آنچه که تئوری بازی تکاملی در مورد یادگیری چند عاملی به ما می گوید

This paper discusses If multi-agent learning is the answer, what is the question? [Y. Shoham, R. Powers, T. Grenager, If multi-agent learning is the answer, what is the question? Artificial Intelligence 171 (7) (2007) 365–377, this issue] from the perspective of evolutionary game theory. We briefly discuss the concepts of evolutionary game theory, and examine the main conclusions from [Y. Shoham, R. Powers, T. Grenager, If multi-agent learning is the answer, what is the question? Artificial Intelligence 171 (7) (2007) 365–377, this issue] with respect to some of our previous work. Overall we find much to agree with, concluding, however, that the central concerns of multiagent learning are rather narrow compared with the broad variety of work identified in [Y. Shoham, R. Powers, T. Grenager, If multi-agent learning is the answer, what is the question? Artificial Inteligence 171 (7) (2007) 365–377, this issue].

In If multi-agent learning is the answer, what is the question? by Shoham, Powers and Grenager [20], the authors make a valiant effort to analyse the state of the field of multiagent learning, to summarise the results that have been achieved within the field, to discern the major research directions that have been followed, and to issue a call to arms. In short, Shoham et al. conclude that most work in multiagent learning can be placed into one of five “buckets” each of which is associated with a distinct research agenda (these descriptions are taken directly from the “caricatures” in Section 5 of [20]): (1) Computational: learning algorithms are a way to compute the properties of a game. (2) Descriptive: learning algorithms describe how natural agents learn in the context of other learners. (3) Normative: learning algorithms give a means to determine which sets of learning rules are in equilibrium with one another. (4) Prescriptive, cooperative: learning algorithms describe how agents should learn in order to achieve distributed control of dynamic systems. (5) Prescriptive, non-cooperative: learning algorithms describe how agents should act to obtain high rewards.Not all work in the field falls into one of these buckets. This means that either we need more buckets, or some work needs to be revisited or reconstructed so as to be well-grounded. The authors also point out that research in multiagent learning is often unclear about which of these agendas it is pursuing—and that, in contrast one needs to be very clear about one’s aims—that the field cannot progress by defining “arbitrary learning strategies and analys(ing) whether the resulting dynamics converge in certain cases to a Nash equilibrium or some other solution concept”, and that the field needs to take evaluation more seriously, especially on some set of standard problems. We basically agree with all of the points that we have quoted above, and below will expand on those that we feel our background leaves us best qualified to discuss. However, we do have one major point of disagreement with the views expressed in [20]. The point we disagree with is the idea that there is some overall taxonomy of research agendas into which all work can be slotted. This is not explicitly stated—all that Shoham et al. say is what we quoted above, that there are five distinct agendas to which some additional ones may need to be added—but the existence of an underlying taxonomy into which these additional categories can be slotted seems to be implied. Our objection is neatly summarised by the following passage from McWhorter’s The Power of Babel: A natural history of language [15] in which the author tries to express how the original language, spoken by our common ancestors, became the many thousands of languages that descended from it. He starts by saying “I have implied that speech varieties have developed like a bush, starting from a single sprout and branching in all directions, each branch then developing subbranches, and so on . . . ” before going on to explain that the inter-relationships between languages, the constant process of adoption of terms from one language into another, and the formation of dialects, creoles and intertwined languages means that [15, page 94]: we might do just as well with another analogy, say stewing a good spring lamb stew without much juice (because the juice messes up the analogy). Clearly, one can distinguish the lamb from the peas from the potatoes from the carrots from the leeks from the rosemary leaves. Yet all of these ingredients, if it’s a good stew, are suffused with juice and flavor from the other items as well. Every now and then, you even encounter a piece of something that, covered with liquid and cooked out of its original shape and consistency, you have to work to figure out the original identity of . . . Overall, there is nothing in this stew that tastes or even looks like it would if you had just dumped it into a pot of boiling water by itself.It seems to us that multiagent learning is such a stew and though it is very helpful to identify the various ingredients—especially if, to stretch the metaphor, some of them would be better taken out of the pot—to concentrate on the constituents misses some of the essence. It is the places in which the agendas that make up the stew meld into new things, things that cannot be put into a taxonomy because they are a mixture, that we often find the most interesting work.1 That said, we should reiterate that we are largely in agreement with the agendas identified in [20], and in the next section amplify our agreement by examining all five agendas through the lens of evolutionary game theory (EGT), which is an area of multiagent learning in which we have been working, and one that is not much discussed in [20]. Following that exploration, we return to our point about the interplay between agendas, illustrating our discussion with some of our recent work.

In this paper we have taken an EGT perspective toward the five research agendas identified by Shoham et al. in [20]. In summary we can say we agree with the existence of the agendas, and the need for greater clarity in work on multiagent learning with respect to how it relates to these agendas, and thus how it contributes to the overall goals of the field. However, we feel that, as in our examples from evolutionary game theory, much work falls across the boundaries of several agendas, contributing to each in some way, and that there is nothing wrong with this.We believe that this “cross-agenda” work is only to be expected, and should be encouraged. Looking at the individual agendas in more detail, we believe that work on the normative agenda has less to contribute to the overall goals of multiagent learning, which are ultimately concerned with computationally limited agents, than the descriptive agenda. It is the latter that will help us to build systems in which real agents and people interact with one another. Because these agents and people will be motivated at some level by their own self-interest, the payoff to individual agents has a more important role to play than equilibrium analysis, though certain classes of equilibria, such as the equilibria in replicator dynamics, are still informative. Putting all of this together, our view is that the core activity of multiagent learning is in the area where Shoham et al.’s first, second and fifth agendas overlap. In this overlap we are concerned with computing the properties (agenda 1) of systems made up of real, rather than idealised, agents (agenda 2), whose decisions are motivated largely by their individual payoffs (agenda 5). It because we believe that multiagent learning is an effective answering to the question of how to construct such systems that makes us interested in the field.