تئوری بازی تکاملی: مفاهیم نظری و برنامه های کاربردی برای اجتماعات میکروبی

Ecological systems are complex assemblies of large numbers of individuals, interacting competitively under multifaceted environmental conditions. Recent studies using microbial laboratory communities have revealed some of the self-organization principles underneath the complexity of these systems. A major role of the inherent stochasticity of its dynamics and the spatial segregation of different interacting species into distinct patterns has thereby been established. It ensures the viability of microbial colonies by allowing for species diversity, cooperative behavior and other kinds of “social” behavior. A synthesis of evolutionary game theory, nonlinear dynamics, and the theory of stochastic processes provides the mathematical tools and a conceptual framework for a deeper understanding of these ecological systems. We give an introduction into the modern formulation of these theories and illustrate their effectiveness focussing on selected examples of microbial systems. Intrinsic fluctuations, stemming from the discreteness of individuals, are ubiquitous, and can have an important impact on the stability of ecosystems. In the absence of speciation, extinction of species is unavoidable. It may, however, take very long times. We provide a general concept for defining survival and extinction on ecological time-scales. Spatial degrees of freedom come with a certain mobility of individuals. When the latter is sufficiently high, bacterial community structures can be understood through mapping individual-based models, in a continuum approach, onto stochastic partial differential equations. These allow progress using methods of nonlinear dynamics such as bifurcation analysis and invariant manifolds. We conclude with a perspective on the current challenges in quantifying bacterial pattern formation, and how this might have an impact on fundamental research in non-equilibrium physics.

This article is intended as an introduction into the concepts and the mathematical framework of evolutionary game theory, seen with the eyes of a theoretical physicist. It grew out of a series of lectures on this topic given at various summer schools in the year 2009.1 Therefore, it is not meant to be a complete review of the field but rather a personal, and hopefully pedagogical, collection of ideas and concepts intended for an audience of advanced students. The first chapter gives an introduction into the theory of games. We will start with “strategic games”, mainly to introduce some basic terminology, and then quickly move to “evolutionary game theory”. The latter is the natural framework for the evolutionary dynamics of populations consisting of multiple interacting species, where the success of a given individual depends on the behavior of the surrounding ones. It is most naturally formulated in the language of nonlinear dynamics, where the game theory terms “Nash equilibrium” or “evolutionary stable strategy” map onto “fixed points” of ordinary nonlinear differential equations. Illustrations of these concepts are given in terms of two-strategy games and the cyclic Lotka–Volterra model, also known as the “rock–paper–scissors” game. We conclude this first chapter by an (incomplete) overview of game-theoretical problems in biology, mainly taken from the field of microbiology. A deterministic description of populations of interacting individuals in terms of nonlinear differential equations, however, misses some important features of actual ecological systems. The molecular processes underlying the interaction between individuals are often inherently stochastic and the number of individuals is always discrete. As a consequence, there are random fluctuations in the composition of the population which can have an important impact on the stability of ecosystems. In the absence of speciation, extinction of species is unavoidable. It may, however, take very long times. Our second chapter starts with some elementary, but very important, notes on extinction times. These ideas are then illustrated for two-strategy games, whose stochastic dynamics is still amenable to an exact solution. Finally, we provide a general concept for defining survival and extinction on an ecological time scale which should be generally applicable. Section 3, introducing the May–Leonard model, serves two purposes. On one hand, it shows how a two-step cyclic competition, split into a selection and reproduction step, modifies the nonlinear dynamics from neutral orbits, as observed in the cyclic Lotka–Volterra model, to an unstable spiral. This teaches that the details of the molecular interactions between individuals may matter and periodic orbits are, in general, non-generic in population dynamics. On the other hand, the May–Leonard model serves to introduce some more advanced concepts from non-linear dynamics: linear stability analysis, invariant manifolds, and normal forms. In addition, the analysis will also serve as a necessary prerequisite for the analysis of spatial games in the subsequent chapter. Section 3 is mainly technical and may be skipped in a first reading. Cyclic competition of species, as metaphorically described by the children’s game “rock–paper–scissors”, is an intriguing motif of species interactions. Laboratory experiments on populations consisting of different bacterial strains of Escherichia coli have shown that bacteria can coexist if a low mobility enables the segregation of the different strains and thereby the formation of patterns [1]. In Section 4 we analyze the impact of stochasticity and an individual’s mobility on the stability of diversity as well as the emerging patterns. Within a spatially-extended version of the May–Leonard model we demonstrate the existence of a sharp mobility threshold, such that diversity is maintained below, but jeopardized above that value. Computer simulations of the ensuing stochastic cellular automaton show that entangled rotating spiral waves accompany biodiversity. These findings are rationalized using stochastic partial differential equations (SPDE), which are reduced to a complex Ginzburg–Landau equation (CGLE) upon mapping the SPDE onto the reactive manifold of the nonlinear dynamics. In Section 5, we conclude with a perspective on the current challenges in quantifying bacterial pattern formation and how this might have an impact on fundamental research in non-equilibrium physics. 1.1. Strategic games Classical Game Theory [2] describes the behavior of rational players. It attempts to mathematically capture behaviors in strategic situations, in which an individual’s success in making choices depends on the choices of others. A classical example of a strategic game is the prisoner’s dilemma. In its classical form, it is presented as follows2: “Two suspects of a crime are arrested by the police. The police have insufficient evidence for a conviction, and, having separated both prisoners, visit each of them to offer the same deal. If one testifies (defects from the other) for the prosecution against the other and the other remains silent (cooperates with the other), the betrayer goes free and the silent accomplice receives the full 10-year sentence. If both remain silent, both prisoners are sentenced to only 1 year in jail for a minor charge. If each betrays the other, each receives a five-year sentence. Each prisoner must choose to betray the other or to remain silent. Each one is assured that the other would not know about the betrayal before the end of the investigation. How should the prisoners act?” The situation is best illustrated in what is called a “payoff matrix” which in the classical formulation is rather a “cost matrix”: Here rows and columns correspond to player (suspect) 1 and 2, respectively. The entries give the prison sentence for player 1; this is sufficient information since the game is symmetric. Imagine you are player 1, and that player 2 is playing the strategy “cooperate”. Then you are obviously better off to play “defect” since you can get free. Now imagine player 2 is playing “defect”. Then you are still better off to defect since 5 years in prison is better than 10 years in prison. If both players are rational players a dilemma arises since both will analyze the situation in the same way and come to the conclusion that it is always better to play “defect” irrespective of what the other suspect is playing. This outcome of the game, both playing “defect”, is called a Nash equilibrium [3]. The hallmark of a Nash equilibrium is that none of the players has an advantage of deviating from his strategy unilaterally. This rational choice, where each player maximizes his own payoff, is not the best outcome! If both defect, they will both be sentenced to prison for 5 years. Each player’s individual reward would be greater if they both played cooperatively; they would both only be sentenced to prison for 1 year. We can also reformulate the prisoner’s dilemma game as a kind of a public good game. A cooperator provides a benefit bb to another individual, at a cost c to itself (with b−c>0b−c>0). In contrast, a defector refuses to provide any benefit and hence does not pay any costs. For the selfish individual, irrespective of whether the partner cooperates or defects, defection is favorable, as it avoids the cost of cooperation, exploits cooperators, and ensures not to become exploited. However, if all individuals act rationally and defect, everybody is, with a gain of 0, worse off compared to universal cooperation, where a net gain of b−c>0b−c>0 would be achieved. The prisoner’s dilemma therefore describes, in its most basic form, the fundamental problem of establishing cooperation. This scheme can be generalized to include other basic types of social dilemmas [4] and [5]. Namely, two cooperators that meet are both rewarded with a payoff RR, while two defectors obtain a punishmentPP. When a defector encounters a cooperator, the first exploits the second, gaining the temptationTT, while the cooperator only gets the sucker’s payoffSS. Social dilemmas occur when R>PR>P, such that cooperation is favorable in principle, while temptation to defect is large: T>ST>S, T>PT>P. These interactions may be summarized by the payoff matrix: Variation of the parameters TT, PP, RR and SS yields four principally different types of games. The prisoner’s dilemma arises if the temptation TT to defect is larger than the reward RR, and if the punishment PP is larger than the suckers payoff SS. As we have already seen above, in this case, defection is the best strategy for the selfish player. Within the three other types of games, defectors are not always better off. For the snowdrift game the temptation TT is still higher than the reward RR but the sucker’s payoff SS is larger than the punishment PP. Therefore, now actually cooperation is favorable when meeting a defector, but defection pays off when encountering a cooperator, and a rational strategy consists of a mixture of cooperation and defection. The snowdrift game derives its name from the potentially cooperative interaction present when two drivers are trapped behind a large pile of snow, and each driver must decide whether to clear a path. Obviously, then the optimal strategy is the opposite of the opponent’s (cooperate when your opponent defects and defect when your opponent cooperates). Another scenario is the coordination game, where mutual agreement is preferred: either all individuals cooperate or defect as the reward RR is higher than the temptation TT and the punishment PP is higher than sucker’s payoff SS. Finally, the scenario of by-product mutualism (also called harmony) yields cooperators fully dominating defectors since the reward RR is higher than the temptation TT and the sucker’s payoff SS is higher than the punishment PP. 1.2. Evolutionary game theory Strategic games, as discussed in the previous section, are thought to be a useful concept in economic and social settings. In order to analyze the behavior of biological systems, the concept of rationality is not meaningful. Evolutionary Game Theory (EGT) as developed mainly by Maynard Smith and Price [6] and [7] does not rely on rationality assumptions but on the idea that evolutionary forces like natural selection and mutation are the driving forces of change. The interpretation of game models in biology is fundamentally different from strategic games in economics or social sciences. In biology, strategies are considered to be inherited programs which control the individual’s behavior. Typically one looks at a population composed of individuals with different strategies who interact generation after generation in game situations of the same type. The interactions may be described by deterministic rules or stochastic processes, depending on the particular system under study. The ensuing dynamic process can then be viewed as an iterative (nonlinear) map or a stochastic process (either with discrete or continuous time). This naturally puts evolutionary game theory in the context of nonlinear dynamics and the theory of stochastic processes. We will see how a synthesis of both approaches helps to understand the emergence of complex spatio-temporal dynamics. In this section, we focus on a deterministic description of well-mixed populations. The term “well-mixed” signifies systems where the individual’s mobility (or diffusion) is so large that one may neglect any spatial degrees of freedom and assume that every individual is interacting with everyone at the same time. This is a mean-field picture where the interactions are given in terms of the average number of individuals playing a particular strategy. Frequently, this situation is visualized as an “urn model”, where two (or more) individuals from a population are randomly selected to play with each other according to some specified game theoretical scheme. The term “deterministic” means that we are seeking a description of populations where the number of individuals Ni(t)Ni(t) playing a particular strategy AiAi is macroscopically large such that stochastic effects can be neglected. Pairwise reactions and rate equations In the simplest setup the interaction between individuals playing different strategies can be represented as a reaction process characterized by some set of rate constants. For example, consider a game where three strategies {A,B,C}{A,B,C} cyclically dominate each other, as in the famous rock–paper–scissors game: AA invades BB, BB outperforms CC, and CC in turn dominates over AA, schematically drawn in Fig. 2: In an evolutionary setting, the game may be played according to an urn model as illustrated in Fig. 1: at a given time tt two individuals from a population with constant size NN are randomly selected to play with each other (react) according to the scheme View the MathML sourceA+B⟶kAA+A, Turn MathJax on equation(1) View the MathML sourceB+C⟶kBB+B, Turn MathJax on View the MathML sourceC+A⟶kCC+C, Turn MathJax on where kiki are rate constants, i.e. probabilities per unit time. This interaction scheme is termed a cyclic Lotka–Volterra model.3 It is equivalent to a set of chemical reactions, and in the deterministic limit of a well-mixed population one obtains rate equations for the frequencies (a,b,c)=(NA,NB,NC)/N(a,b,c)=(NA,NB,NC)/N: ∂ta=a(kAb−kCc),∂ta=a(kAb−kCc), Turn MathJax on equation(2) ∂tb=b(kBc−kAa),∂tb=b(kBc−kAa), Turn MathJax on ∂tc=c(kCa−kBb).∂tc=c(kCa−kBb). Turn MathJax on Here the right hand sides gives the balance of “gain” and “loss” processes. The phase space of the model is the simplex S3S3, where the species’ densities are constrained by a+b+c=1a+b+c=1. There is a constant of motion for the rate equations, Eq. (2), namely the quantity ρ≔akBbkCckAρ≔akBbkCckA does not evolve in time [10]. As a consequence, the phase portrait of the dynamics, shown in Fig. 3, yields cycles around the reactive fixed point.The concept of fitness and replicator equations Another route of setting up an evolutionary dynamics, often taken in the mathematical literature of evolutionary game theory [11] and [10], introduces the concept of fitness and then assumes that the per-capita growth rate of a strategy AiAi is given by the surplus in its fitness with respect to the average fitness of the population. We will illustrate this reasoning for two-strategy games with a payoff matrix Let NANA and NBNB be the number of individuals playing strategy AA and BB in a population of size N=NA+NBN=NA+NB. Then the relative abundances of strategies AA and BB are given by equation(4) View the MathML sourcea=NAN,b=NBN=(1−a). Turn MathJax on The “fitness” of a particular strategy AA or BB is defined as a constant background fitness, set to 1, plus the average payoff obtained from playing the game: equation(5) fA(a)≔1+Ra+S(1−a),fA(a)≔1+Ra+S(1−a), Turn MathJax on equation(6) fB(a)≔1+Ta+P(1−a).fB(a)≔1+Ta+P(1−a). Turn MathJax on In order to mimic an evolutionary process one is seeking a dynamics which guarantees that individuals using strategies with a fitness larger than the average fitness increase while those using strategies with a fitness below average decline in number. This is, for example, achieved by choosing the per-capita growth rate, ∂ta/a∂ta/a, of individuals playing strategy AA proportional to their surplus in fitness with respect to the average fitness of the population, equation(7) View the MathML sourcef̄(a)≔afA(a)+(1−a)fB(a). Turn MathJax on The ensuing ordinary differential equation is known as the standard replicator equation [11] and [10] equation(8) View the MathML source∂ta=[fA(a)−f̄(a)]a. Turn MathJax on Lacking a detailed knowledge of the actual “interactions” of individuals in a population, there is, of course, plenty of freedom in how to write down a differential equation describing the evolutionary dynamics of a population. Indeed, there is another set of equations frequently used in EGT, called adjusted replicator equations, which reads equation(9) View the MathML source∂ta=fA(a)−f̄(a)f̄(a)a. Turn MathJax on Here we will not bother to argue why one or the other is a better description. As we will see later, these equations emerge quite naturally from a full stochastic description in the limit of large populations. 1.3. Nonlinear dynamics of two-player games This section is intended to give a concise introduction into elementary concepts of nonlinear dynamics [12]. We illustrate those for the evolutionary dynamics of two-player games characterized in terms of the payoff matrix, Eq. (3), and the ensuing replicator dynamics equation(10) View the MathML source∂ta=a(fA−f̄)=a(1−a)(fA−fB). Turn MathJax on This equation has a simple interpretation: the first factor, a(1−a)a(1−a), is the probability for AA and BB to meet and the second factor, fA−fBfA−fB, is the fitness advantage of AA over BB. Inserting the explicit expressions for the fitness values one finds equation(11) ∂ta=a(1−a)[μA(1−a)−μBa]≕F(a),∂ta=a(1−a)[μA(1−a)−μBa]≕F(a), Turn MathJax on where μAμA is the relative benefit of AA playing against BB and μBμB is the relative benefit of BB playing against AA: equation(12) View the MathML sourceμA≔S −P,μB≔T −R. Turn MathJax on Hence, as far as the replicator dynamics is concerned, we may replace the payoff matrix by Eq. (11) is a one-dimensional nonlinear first-order differential equation for the fraction aa of players AA in the population, whose dynamics is most easily analyzed graphically. The sign of F(a)F(a) determines the increase or decrease of the dynamic variable aa; compare the right half of Fig. 4. The intersections of F(a)F(a) with the aa-axis (zeros) are fixed points, a∗a∗. Generically, these intersections are with a finite slope F′(a∗)≠0F′(a∗)≠0; a negative slope indicates a stable fixed point while a positive slope an unstable fixed point. Depending on some control parameters, here μAμA and μBμB, the first or higher order derivatives of FF at the fixed points may vanish. These special parameter values mark “threshold values” for changes in the flow behavior (bifurcations) of the nonlinear dynamics. We may now classify two-player games as summarized in Table 1. For the prisoner’s dilemma μA=−c<0μA=−c<0 and μB=c>0μB=c>0 and hence players with strategy BB (defectors) are always better off (compare the payoff matrix). Both players playing strategy BB is a Nash equilibrium. In terms of the replicator equations this situation corresponds to F(a)<0F(a)<0 for a≠0a≠0 and F(a)=0F(a)=0 at View the MathML sourcea=0,1 such that a∗=0a∗=0 is the only stable fixed point. Hence the term “Nash equilibrium” translates into the “stable fixed point” of the replicator dynamics (nonlinear dynamics). For the snowdrift game both μA>0μA>0 and μB>0μB>0 such that F(a)F(a) can change sign for a∈[0,1]a∈[0,1]. In fact, View the MathML sourceaint∗=μA/(μA+μB) is a stable fixed point while View the MathML sourcea∗=0,1 are unstable fixed points; see the right panel of Fig. 4. Inspection of the payoff matrix tells us that it is always better to play the opposite strategy of your opponent. Hence there is no Nash equilibrium in terms of the pure strategies AA or BB. This corresponds to the fact that the boundary fixed points View the MathML sourcea∗=0,1 are unstable. There is, however, a Nash equilibrium with a mixed strategy where a rational player would play strategy AA with probability pA=μA/(μA+μB)pA=μA/(μA+μB) and strategy BB with probability pB=1−pApB=1−pA. Hence, again, the term “Nash equilibrium” translates into the “stable fixed point” of the replicator dynamics. For the coordination game, there is also an interior fixed point at View the MathML sourceaint∗=μA/(μA+μB), but now it is unstable, while the fixed points at the boundaries View the MathML sourcea∗=0,1 are stable. Hence we have bistability: for initial values View the MathML sourcea<aint∗ the flow is towards a=0a=0 while it is towards a=1a=1 otherwise. In the terminology of strategic games there are two Nash equilibria. The game harmony corresponds to the prisoner’s dilemma with the roles of AA and BB interchanged. 1.4. Bacterial games Bacteria often grow in complex, dynamical communities, pervading the earth’s ecological systems, from hot springs to rivers and the human body [13]. As an example, in the latter case, they can cause a number of infectious diseases, such as lung infection by Pseudomonas aeruginosa. Bacterial communities, quite generically, form biofilms [13] and [14], i.e., they arrange into a quasi-multi-cellular entity where they interact highly. These interactions include competition for nutrients, cooperation by providing various kinds of public goods essential for the formation and maintenance of the biofilm [15], communication through the secretion and detection of extracellular substances [16] and [17], and last but not least physical forces. The ensuing complexity of bacterial communities has conveyed the idea that they constitute a kind of “social groups” where the coordinated action of individuals leads to various kinds of system-level functionalities. Since additionally microbial interactions can be manipulated in a multitude of ways, many researchers have turned to microbes as the organisms of choice to explore fundamental problems in ecology and evolutionary dynamics. Two of the most fundamental questions that challenge our understanding of evolution and ecology are the origin of cooperation [18], [19], [20], [16], [17], [21], [22] and [23] and biodiversity [24], [25], [26], [1] and [27]. Both are ubiquitous phenomena yet are conspicuously difficult to explain since the fitness of an individual or the whole community depends in an intricate way on a plethora of factors, such as spatial distribution and mobility of individuals, secretion and detection of signaling molecules, toxin secretion leading to inter-strain competition and changes in environmental conditions. It is fair to say that we are still a long way off from a full understanding, but the versatility of microbial communities makes their study a worthwhile endeavor with exciting discoveries still ahead of us. Cooperation Understanding the conditions that promote the emergence and maintenance of cooperation is a classic problem in evolutionary biology [28], [29] and [7]. It can be stated in the language of the prisoner’s dilemma. By providing a public good, cooperative behavior would be beneficial for all individuals in the whole population. However, since cooperation is costly, the population is at risk from invasion by “selfish” individuals (cheaters), who save the cost of cooperation but can still obtain the benefit of cooperation from others. In evolutionary theory many principles were proposed to overcome this dilemma of cooperation: repeated interaction [20] and [28], punishment [20] and [30], or kin discrimination [31] and [32]. All of these principles share one fundamental feature: They are based on some kind of selection mechanism. Similar to the old debate between “selectionists” and “neutralists” in evolutionary theory [33], there is an alternative. Due to random fluctuations, a population initially composed of both cooperators and defectors may (with some probability) become fixed in a state of cooperators only [34]. We will come back to this point later in Section 2.2. There has been an increasing number of experiments using microorganisms to shed new light on the problem of cooperation [18], [19], [22] and [23]. Here, we will shortly discuss a recent experiment on “cheating in yeast” [22]. Budding yeast prefers to use the monosaccharides glucose and fructose as carbon sources. If they have to grow on sucrose instead, the disaccharide must first be hydrolyzed by the enzyme invertase. Since a fraction of approximately 1−ϵ=99%1−ϵ=99% of the produced monosaccharides diffuses away and is shared with neighboring cells, it constitutes a public good available to the whole microbial community. This makes the population susceptible to invasion by mutant strains that save the metabolic cost of producing invertase. One is now tempted to conclude from what we have discussed in the previous sections that yeast is playing the prisoner’s dilemma game. The cheater strains should take over the population and the wild type strain should become extinct. But, this is not the case. Gore and collaborators [22] show that the dynamics is rather described by a snowdrift game, in which cheating can be profitable, but is not necessarily the best strategy if others are cheating too. The explanation given is that the growth rate as a function of glucose is highly concave and, as a consequence, the fitness function is non-linear in the payoffs4 equation(13) fC(x)≔[ϵ+x(1−ϵ)]α−c,fC(x)≔[ϵ+x(1−ϵ)]α−c, Turn MathJax on equation(14) fD(a)≔[x(1−ϵ)]α,fD(a)≔[x(1−ϵ)]α, Turn MathJax on with α≈0.15α≈0.15 determined experimentally. The ensuing phase diagram, Fig. 5, as a function of capture efficiency ϵϵ and metabolic cost cc shows an altered intermediate regime with a bistable phase portrait, i.e. the hallmark of a snowdrift game as discussed in the previous section. This explains the experimental observations. The lesson to be learned from this investigation is that defining a payoff function is not a trivial matter, and a naive replicator dynamics fails to describe biological reality. It is, in general, necessary to have a detailed look on the nature of the biochemical processes responsible for the growth rates of the competing microbes.Pattern formation Investigations of microbial pattern formation have often focussed on one bacterial strain [35], [36] and [37]. In this respect, it has been found that bacterial colonies on substrates with a high nutrient level and intermediate agar concentrations, representing “friendly” conditions, grow in simple compact patterns [38]. When instead the level of nutrient is lowered, when the surface on which bacteria grow possesses heterogeneities, or when the bacteria are exposed to antibiotics, complex, fractal patterns are observed [35], [39] and [40]. Other factors that affect the self-organizing patterns include motility [41], the kind of bacterial movement, e.g., swimming [42], swarming, or gliding [43] and [44], as well as chemotaxis and external heterogeneities [45]. Another line of research has investigated patterns of multiple co-evolving bacterial strains. As an example, recent studies looked at growth patterns of two functionally equivalent strains of E. coli and showed that, due to fluctuations alone, they segregate into well-defined, sector-like regions [36] and [46]. The Escherichia Col E2 system Several Colibacteria such as E. coli are able to produce and secrete specific toxins called Colicines that inhibit the growth of other bacteria. Kerr and coworkers [1] have studied three strains of E. coli, amongst which one is able to produce the toxin Col E2 that acts as an DNA endonuclease. This poison producing strain (C) kills a sensitive strain (S), which outgrows the third, resistant one (R), as resistance bears certain costs. The resistant bacteria grow faster than the poisonous ones, as the latter are resistant and produce a poison, which is yet an extra cost. Consequently, the three strains of E. coli display a cyclic competition, similar to the children’s game rock–paper–scissors. When placed on a Petri-dish, all three strains coexist, arranging in time-dependent spatial clusters dominated by one strain. In Fig. 6, snapshots of these patterns monitored over several days are shown. Sharp boundaries between different domains emerge, and all three strains co-evolve at comparable densities. The patterns are dynamic: Due to the non-equilibrium character of the species’ interactions, clusters dominated by one bacterial strain cyclically invade each other, resulting in an endless hunt of the three species on the Petri-dish. The situation changes considerably when putting the bacteria in a flask with additional stirring. Then, only the resistant strain survives, while the two others die out after a short transient time.These laboratory experiments thus provide intriguing experimental evidence for the importance of spatial patterns for the maintenance of biodiversity. In this respect, many further questions regarding the spatio-temporal interactions of competing organisms under different environmental conditions lie ahead. Spontaneous mutagenesis of single cells can lead to enhanced fitness under specific environmental conditions or due to interactions with other species. Moreover, interactions with other species may allow unfit, but potentially pathogenic bacteria to colonize certain tissues. Additionally, high concentrations of harmless bacteria may help pathogenic ones to nest on tissues exposed to extremely unfriendly conditions. Information about bacterial pattern formation arising from bacterial interaction may therefore allow one to develop mechanisms to avoid pathogenic infection.

In these lecture notes we have given an introduction into evolutionary game theory. The perspective we have taken was that starting from agent-based models, the dynamics may be formulated in terms of a hierarchy of theoretical models. First, if the population size is large and the population is well-mixed, a set of ordinary differential equations can be employed to study the system’s dynamics and ensuing stationary states. Game theoretical concepts of “equilibria” then map to “attractors” of the nonlinear dynamics. Setting up the appropriate dynamic equations is a non-trivial matter if one is aiming at a realistic description of a biological system. For instance, as nicely illustrated by a recent study on yeast [22], a linear replicator equation might not be sufficient to describe the frequency-dependence of the fitness landscape. We suppose that this is rather the rule than the exception for biological systems such as microbial populations. Second, for well-mixed but finite populations, one has to account for stochastic fluctuations. Then there are two central questions: (i) What is the probability of a certain species to go extinct or to become fixated in a population? (ii) How long does this process take? These questions have to be answered by employing concepts from the theory of stochastic processes. Since most systems have absorbing states, we have found it useful to classify the stability of a given dynamic system according to the scaling of the expected extinction time with population size. Third and finally, taking into account the finite mobility of individuals in an explicit spatial model, a description in terms of stochastic partial differential equations becomes necessary. These Langevin equations describe the interplay between reactions, diffusion and noise, which give rise to a plethora of new phenomena. In particular, spatio-temporal patterns or, more generally, spatio-temporal correlations, may emerge which can dramatically change the ecological and evolutionary stability of a population. For non-transitive dynamics, like the rock–scissors–paper game played by some microbes [1], there is a mobility threshold which demarcates regimes of maintenance and loss of biodiversity [73]. Since, for the rock–scissors–paper game, the nature of the patterns and the transition was encoded in the flow of the nonlinear dynamics on the reactive manifold, one might hope that a generalization of the outlined approach might be helpful in classifying a broader range of game-theoretical problems and identify some “universality classes”. What are the ideal experimental model systems for future studies? We think that microbial populations will play a major role since interactions between different strains can be manipulated in a multitude of ways. In addition, experimental tools like microfluidics and various optical methods allow for easy manipulation and observation of these systems, from the level of an individual up to the level of a whole population. Bacterial communities represent complex and dynamic ecological systems. They appear in the form of free-floating bacteria as well as biofilms in nearly all parts of our environment, and are highly relevant for human health and disease [123]. Spatial patterns arise from heterogeneities of the underlying “landscape” or self-organized by the bacterial interactions, and play an important role in maintaining species diversity [27]. Interactions comprise, amongst others, competition for resources and cooperation by sharing of extracellular polymeric substances. Another aspect of interactions is chemical warfare. As we have discussed, some bacterial strains produce toxins such as colicin, which acts as a poison to sensitive strains, while other strains are resistant [1]. Stable coexistence of these different strains arises when they can spatially segregate, resulting in self-organizing patterns. There is a virtually inexhaustible complexity in the structure and dynamics of microbial populations. The recently proposed term “socio-microbiology” [124] expresses this notion in a most vivid form. Investigating the dynamics of those complex microbial populations in a challenging interdisciplinary endeavor, which requires the combination of approaches from molecular microbiology, experimental biophysical methods and theoretical modeling. The overall goal would be to explore how collective behavior emerges and is maintained or destroyed in finite populations under the action of various kinds of molecular interactions between individual cells. Both communities, biology as well as physics, will benefit from this line of research. Stochastic interacting particle systems are a fruitful testing ground for understanding generic principles in non-equilibrium physics. Here biological systems have been a wonderful source of inspiration for the formulation of new models. For example, MacDonald [125] looking for a mathematical description for mRNA translation into proteins managed by ribosomes, which bind to the mRNA strand and step forward codon by codon, formulated a non-equilibrium one-dimensional transport model, nowadays known as the totally asymmetric simple exclusion process. This model has led to significant advances in our understanding of phase transitions and the nature of stationary states in non-equilibrium systems [126] and [127]. Searching for simplified models of epidemic spreading without immunization Harris [128] introduced the contact process. In this model infectious individuals can either heal themselves or infect their neighbors. As a function of the infection and recovery rate it displays a phase transition from an active to an absorbing state, i.e. the epidemic disease may either spread over the whole population or vanish after some time. The broader class of absorbing-state transitions has recently been reviewed [129]. Another well studied model is the voter model where each individual has one of two opinions and may change it by imitation of a randomly chosen neighbor. This process mimics in a naive way opinion making [130]. Actually, it was first considered by Clifford and Sudbury [131] as a model for the competition of species and only later named the voter model by Holley and Liggett [132]. It has been shown rigorously that on a regular lattice there is a stationary state where two “opinions” coexist in systems with spatial dimensions where the random walk is not recurrent [133] and [130]. A question of particular interest is how opinions or strategies may spread in a population. In this context it is important to understand the coarsening dynamics of interacting agents. For a one-dimensional version of the the rock–paper–scissors game, Frachebourg and collaborators [134] and [135] have found that starting from some random distribution, the species organize into domains that undergo (power law) coarsening until finally one species takes over the whole lattice. Including mutation, the coarsening process is counteracted and by an interesting interplay between equilibrium and non-equilibrium processes a reactive stationary state emerges [136]. The list of interesting examples, of course, continues and one may hope that in the future there will be an even more fruitful interaction between biologically relevant processes and basic research in non-equilibrium dynamics.