Sept 2017
# Marc Harper, Ph.D.

Contact: marc@marcharper.net

*Reinforcement Learning Produces Dominant Strategies for the Iterated Prisoner's Dilemma*, with Vincent Knight, Martin Jones, Georgios Koutsovoulos, Nikoleta E. Glynatsi, and Owen Campbell. [ArXiv preprint] (2017)

We use reinforcement learning (evolutionary algorithms) and a variety of machine learning techniques to produce strategies that win iterated prisoner's dilemma tournaments.*Evolution Reinforces Cooperation with the Emergence of Self-Recognition Mechanisms: an empirical study of the Moran process for the iterated Prisoner's dilemma*, with Vincent Knight, Nikoleta E. Glynatsi, and Owen Campbell. [ArXiv preprint] (2017)

Using reinforcement learning techniques we show that agents naturally evolve handshaking mechanisms to resist invasion. We also use various machine learning techniques to produce highly capable invaders, and given a comprehensive computational study of fixation probabilities of the Moran process for ~200 strategies.*An open reproducible framework for the study of the iterated prisoner's dilemma*, with Vince Knight, Owen Campbell, Karol Langner, et al.**Journal of Open Research Software**(open access) [ArXiv preprint] (2016).

The Axelrod library is an open source Python package that allows for reproducible game theoretic research for the Iterated Prisoner's Dilemma-
*Stationary Stability for Evolutionary Dynamics in Finite Populations*, with Dashiell Fryer.**Entropy**(open access) [ArXiv preprint] (2016)

We show that the maxima and minima of the Moran process satisfy an analog of evolutionary stability (incorporating mutation), generalizing the Lyapunov theory of the replicator equation to finite population Markov processes with mutation. See this gallery of examples (Bomze's 3x3 archetypes) for a demonstration. More precisely, we show that the stationary distribution of the Moran process (and related processes) with mutation in finite populations contains information about the evolutionary stability of states of the underlying process. *Entropic Equilibria Selection of Stationary Extrema in Finite Populations*, with Dashiell Fryer. [ArXiv preprint] (2015)

We use the stationary distribution and entropy rates of the Moran process with mutation to compare equilibria within a Markov process and across similar Markov processes. Altering the strength of selection, mutation rate, or population size can change which equilibria is most likely under the Moran process with mutation.*The Art of War: Beyond Memory-one Strategies in Population Games*, with Chris Lee and Dashiell Fryer.**PLoS One**[ArXiv preprint] (2015)

We present a highly-robust machine learning-based strategy for the prisoner's dilemma in population games that naturally forms coalitions, is typically able to invade any other opponent (more often than a neutral mutant), and is highly-resistant to invasion by other strategies.*Lyapunov Functions for Time-Scale Dynamics on Riemannian Geometries of the Simplex*, with Dashiell Fryer.**Dynamic Games and Applications (DGAA)**(2014) preprint-pdf (Formerly titled "Stability of Evolutionary Dynamics on Time Scales", ArXiv preprint)

We give a far-reaching Lyapunov theorem for incentive dynamics on time-scales for a large class of Riemannian geometries, with a wealth of examples. This work substantially generalizes the results in my 2011 Physica D paper "Escort Evolutionary Game Theory". An overview is given in this Azimuth blog post.*Entropy Rates of the Multidimensional Moran Processes and Generalizations*[ArXiv preprint] (2014)

Higher-dimensional followup to "The Inherent Randomness of Evolving Populations".*The Inherent Randomness of Evolving Populations*,**Physical Review E.**[ArXiv preprint] (2013)

Computations and theorems on the entropy rates of the Moran and Wright-Fisher processes with mutations.*Escort Evolutionary Game Theory*,**Physica D**, Vol 240, Issue 18 [ArXiv preprint] (2011, 30+ citations, originally part of my 2009 PhD thesis)

This paper explores the evolutionary dynamics of generalized entropies and information divergences, simultaneously deriving Lyapunov functions for an infinite family of dynamics that includes the replicator and projection dynamics. (*Information Geometry and Evolutionary Game Theory*. [ArXiv preprint] (2009, 20+ citations, part of my PhD thesis)*The Replicator Equation as a Continuous Inference Dynamic*. [ArXiv preprint] (2009, 20+ citations, part of my PhD thesis)

Replicator dynamics and Bayesian inference are closely related. Cosma Shalizi independently found a similar relationship, see Appendix A of his paper Dynamics of Bayesian Updating with Dependent Data and Misspecifed Models

*Comprehensive Detection of Genes Causing a Phenotype using Phenotype Sequencing and Pathway Analysis*, with Luisa Gronenberg, James Liao, and Chris Lee.**PLoS One**[ArXiv preprint]

We enhanced the phenotype sequencing method with gene pathway and functional gene association databases for a large increase in detection power. (2014)*Genome-wide Analysis of Mutagenesis Bias and Context Sensitivity of N-methyl-N'-nitro-N-nitrosoguanidine (NTG)*, with Chris Lee.**Mutation Research**, Volume 731, Issues 1-2, 1 March 2012, Pages 64-67. (2012) 19+ citations*Phenotype sequencing: identifying the genes that cause a phenotype directly from pooled sequencing of independent mutants*with Zugen Chen, Tracy Toy, Lara Machado, Stan Nelson, James Liao, and Chris Lee.**PLoS ONE**. Phenoseq at github | (2011) 18+ citations

*Mathematics Placement at the University of Illinois*, with Alison Ahlgren Reddy.**PRIMUS**, Volume 23, Issue 8, pages 683-702 post-print pdf

An overview of the data from the successful placement program from the University of Illinois. (2013)- Contributed Chapter,
*ALEKS-based Placement at the University of Illinois*, in*Knowledge Spaces: Applications in Education*, with Alison Ahlgren Reddy (2013) *Assessment and Placement through Calculus I at the University of Illinois*, with Alison Ahlgren Reddy.**Notices of the AMS**. (2011)

*Characterizing Finite Population Dynamics via Information Theory*, Workshop on Evolution, Statistical Mechanics and Information Theory, Parmenides Foundation, Munich. [slides] (July 2016)*Information Transport and Evolutionary Dynamics*, NIMBioS, Entropy and Information in Biological Systems [slides] (April 2015)*Equilibrium Selection for Markov Processes via Random Trajectory Entropy with applications to Finite Population Biology*, with Dashiell Fryer, Joint Math 2015 [slides] (January 2015)*Characterizations of Stationary Extrema with Applications to Finite Population Models*, with Dashiell Fryer, Joint Math 2015 [slides] (January 2015)*A Powerful Long Memory Strategy for the Prisoner's Dilemma, with Dashiell Fryer and Chris Lee*, with Dashiell Fryer and Chris Lee, Joint Math 2015 [slides] (January 2015)*Stationary Stability for Evolutionary Dynamics in Finite Populations*[abstract], with Dashiell Fryer, SIAM Conference on the Life Sciences (August 2014)*Quantifying the Relationships among Natural Selection, Mutation, and Stochastic Drift in Multidimensional Finite Populations*[abstract], SIAM Conference on the Life Sciences (August 2014)*A Novel Strategy That Dominates Zero Determinant and Other Known Strategies in Multiplayer Evolutionary Games*[abstract], with Chris Lee and Dashiell Fryer, SIAM Conference on the Life Sciences (August 2014)*Stationary Stability for Evolutionary Dynamics in Finite Populations*[abstract], with Dashiell Fryer, SIAM Annual Meeting (July 2014)*Time Scale Calculus and Evolutionary Dynamics*[abstract], with Dashiell Fryer, SIAM Annual Meeting (July 2014)*Quantifying the Relationships among Natural Selection, Mutation, and Stochastic Drift in Multidimensional Finite Populations*[abstract], SIAM Annual Meeting (July 2014)*The Inherent Randomness of Evolving Populations*[abstract], JMM (Jan 2014)*Evolutionary Stability in Finite Populations*[abstract], with Dashiell Fryer, JMM (Jan 2014)*Stationary Stability in Finite Populations*, UCLA Probability Theory Seminar slides [pdf] (Dec 2013)*Time-scale Lyapunov functions for Incentive Dynamics on Riemannian Geometries*, with Dashiell Fryer, SIAM DS (May 2013)*Phenotype Sequencing: Identifying the Genes and Pathways that cause a Phenotype using EcoCyc*, Conference on Predicting Cell Metabolism and Phenotypes slides [pdf] (March 2013)*Inferring Fitness in Finite, Variably-Sized, Evolving, and Dynamically-Structured Populations*, Joint Mathematics Meeting slides [pdf] (Jan 2013)*Time-scale Lyapunov functions for Incentive Dynamics on Riemannian Geometries, with Dashiell Fryer, Joint Mathematics Meeting*slides [pdf] (January 2013)*Quartet partition tests for O(N log N) tree construction*, Bioinformatics Retreat, UCLA (February 2012)*Phenotype Sequencing, Joint Mathematics Meeting*(January 2012)*Information Theory and Geometry in Evolutionary Dynamics*, Info-Evo Seminar, UCLA (November 2011)*Phenotype Sequencing*, IPAM Workshop, UCLA (October 2011)*Information Gradients and Evolutionary Dynamics*, Seminar at the University of Illinois (July 2010)

*Precalculus Redesign: The Influence of a Placement Program and the Power of a Name*, with Alison Reddy, MAA Mathfest (August 2014)*Panel discussion on online education in mathematics at the Missouri MAA meeting, presentation of results from the placement program at the University of Illinois at a conference in Connecticut. (March 2014)**Identifying Concepts Critical for Success in Calculus at the University of Illinois*[abstract], with Alison Ahlgren Reddy, JMM (Jan 2014)*Identifying Crucial Mathematics Concepts and Skills for Course Success*, with Alison Reddy, AMATYC (Nov 2013)*The Impact of Arithmetic Skills on Success in Calculus II and III*, with Alison Ahlgren Reddy, Joint Mathematics Meetings (Jan 2013)*Connecting student knowledge and course performance at the University of Illinois.*Joint Mathematics Meetings (Jan 2012)*Identifying Crucial Concepts and Skills for Success in College Algebra through Calculus*, with Alison Ahlgren Reddy. Mathfest (Aug 2011)*Connecting student knowledge and course performance at the University of Illinois*, with Alison Ahlgren Reddy. Joint Mathematics Meetings (Jan 2011)*Readiness Assessment and Course Placement through Introductory Calculus*, with Alison Ahlgren Reddy. Joint Mathematics Meetings (Jan 2010)