هوش مصنوعی و سازمان های سیاسی: اکتشاف بر اساس کار علمی تخیلی Iain M. Banks

This paper, using science fiction as a heuristic support for exploring technical potentialities, is based on part of the works of Iain M. Banks, the novels of the “Culture series”, in order to examine the role of artificial intelligences and the effects they could have on the life of a community from a political point of view. This series of science fiction novels portrays a galactic civilization based on anarchistic principles in which intelligent machines are largely responsible for managing the tasks linked to the handling of community affairs, thus freeing up the population to pursue more spiritual or fun activities. The first part of this paper shows that beyond the elements included in the stories, the Culture novels can be a way to address political questions that are raised by the widespread presence of highly evolved machines in the organization of a society. The second part, which takes into consideration the supposed founding principles of this civilization, examines the anarchist thought in order not only to display the correspondences between this thought and the vision of Iain M. Banks, but also to show that the various anarchistic currents are in a way outdistanced by the emerging challenges posed by these novels. The third part, written again from a political standpoint, attempts to establish more concrete connections, based on discernable evolutions in computerization or automation of technological systems, which seem to be working their way into a growing number of social processes and their regulation.

Through analyses combining a literary perspective, political theory and forward thinking, the objective of this contribution is to test a hypothesis that sounds like science fiction, but could go beyond science fiction itself. This hypothesis is based on part of the works of the Scottish writer Iain M. Banks, notably the “Culture novels”, and the social organization that is described in these works. This series of novels portrays a galactic civilization ruled by anarchistic principles in which problems of shortage are overcome and power structures seem to be dismantled. In this civilization called the Culture, artificial intelligences or “Minds” are responsible for managing the tasks linked to the handling of collective affairs, thus freeing up the population to pursue more spiritual or fun activities. The type of social organization described by Iain M. Banks in his novels1 exists mostly because of the protective support provided by these artificial intelligences. If this hypothesis is considered beyond a literary point of view, could it help to conceive the role that “intelligent” machines, or at least highly evolved ones, might play in social and political organization? How could these machines be integrated into collective life? How far would their inclusion go concerning their ability to modify institutional workings? In the civilization model of the Culture, certain ontological distinctions have disappeared, since these entities behave like people and are treated as such. Vessels and space stations have their own “Minds” that make their own choices. In a way, these “conscious” and “sensitive” machines, which are much more intelligent than humans, “are” these spacecrafts. They are the reflective backbone of the Culture, which in fact they control more than they actually live there. If we follow the vision of Iain M. Banks, the development and the widespread presence of these artificial intelligences have disrupted the political system, and even the conception of politics. This would be a very specific application of anarchistic principles. The author has indeed created an organized world in which the plan to replace the government of men with the administration of things has been carried out, thanks to artificial intelligences and limitless material wealth and energy. In this model, there would not really be any political choices left to be made.2 Difficult decisions brought about by resource allocation problems would have no reason to be, or at worst could be resolved using enhanced processing power. Abuse of power would not really be feared, since power would in a way be allocated to these artificial intelligences, which, constitutively, would have risen above these challenges (or in any case, for who this type of temptation would be senseless). In the work of Iain M. Banks, these elements are not simple elements of science fiction scenery: they play an important and intimate role in the stories. Going beyond literary analysis, they can be used as a basis of questioning regarding the possibilities of “social” regulation without direct human intervention, or more precisely, with the mediation of machines evolving towards a form of artificial intelligence. This contribution will also be a way to test up to what point and on what basis such a hypothesis can hold up. Can advances in computer technologies lead to re-imagining the possible ways societies can be regulated?3 If so, to what extent? What is left of politics when it becomes dependent on computer systems that are more and more advanced? It is difficult to answer these questions without finding (and it is also one of the challenges of this article) how to initiate a discussion about techniques which do not exist in practice, or which exist only in a potential state. The solution proposed here is to consider future-oriented fictional works as heuristic media.4 More precisely, works of science fiction can be taken both as a reservoir of thought experiments and as forms of problematizations (in the sense of Michel Foucault). These works may not have been conceived as thought experiments, but the majority of them can be considered according to this model, in particular by providing hypotheses to work with (What if…?). Science fiction can be considered jointly as a way to problematize not only developments in the field of science (if one remains attached to the name of the genre), but also, and perhaps above all, more or less direct consequences on social and political systems. In the manner of Michel Foucault,5 these problematizations can be conceived as ways for thought to seize objects which have a relatively new appearance. To be more precise, they may be ways to examine conditions of possibility and function so that between the beginning and the end of the work of fiction, the representation of a question is changed. In the case of a literary work, these problematizations may also merely be the rearrangement of diffuse representations more or less consciously taken up by the author. Reconsidered in this manner, the fictional material can find methodological support to also become a medium of knowledge, even if its content may seem very detached from reality. On the subject at hand, it can consist in identifying narrative situations in which the reader can see these “artificial intelligences” operating. These representations are scattered, but their combination is expected to draw a relatively coherent configuration,6 with the added benefit of intellectual freedom enabling to go beyond the question of the (largely artificial) borders between what is technical and what is political. Even if the envisaged technologies are still hypothetical, potentialities can thus be actualized, not in reality, but in a fictional construction (which can happen to create effects of reality). Treated as a form of problematization (also with its share of reactivation of more or less ancient myths), science fiction can then be more easily related to other forms of problematization, such as those which are available in political or philosophical reflections, or those which weave the discursive accompaniment of technical developments. To progress in this type of questioning, it is no longer possible to maintain a vision of machines from the last century. That would mean misunderstanding the challenges that might come up in the more or less near future. It is necessary to “take out the notion of machine from an industrial conception”, to quote an objective set by Frédéric Vengeon in a syllabus from the Collège international de philosophie. 7 And above all, it is necessary to restore not only the mechanical nature of machines, but also the digital one. 8 In fact, this evolution of machines appears to generate important consequences. For a long time, humanity has filled the world with various artifacts, but it now seems in the process of adding a whole range of artifacts with new possibilities. In addition to machines transforming energy and/or material, there are now machines capable of processing information flows, and this in quantities and at speeds that seem prodigious. It is still unclear whether these machinic artifacts might be the premise of “artificial intelligences” to come. In any case, these artifacts can no longer be considered as mere instruments for human activities, but they eventually form a socio-technical assemblage in which they appear to gain more self-regulatory capacities. In this evolution, the relationship of the collective with its artefactual creations is likely to change. Of course, non-human artifacts could already benefit from forms of delegation from humans. But with “artificial intelligence”, the stakes could rise to a higher level. Above all, such an issue is typically part of issues that may come from an increase in technological development and that has become difficult to conceive in the present tense (the risk being to allow these changes without thinking about their political implications). Although these potential advances in technology are not yet in operation, it is thus useful to find a way to put them in context. Notwithstanding, given the uncertainties in the possible developments, it is better not to have a too restrictive definition of “artificial intelligences”, and consider them as openly as Iain M. Banks does. It is preferable to not define them according to what they are (that is, an essentialist vision), but what they do or rather what they seem to be able to do (that is, a pragmatic vision), thus considering the web of relations they are a part of. This contribution, which thus uses the work of fiction as a way to reflect on the social and political inclusion of technological evolutions,9 will be organized into three sections. The first part will show the political questions that the novels, which have the Culture as a framework, allowed to be formulated regarding the consequences of progress in artificial intelligence and their effects on the organization of societies. The second part, which takes into consideration the supposed founding principles of this civilization, examines the anarchist thought in order not only to display the correspondences between this thought and the vision of Iain M. Banks, but also to show that the various anarchistic currents are in a way outdistanced by the emerging challenges posed by these novels. The third part, written again from a political standpoint, attempts to establish more concrete connections, based on discernable evolutions in computerization or automation of technological systems, which can participate in the regulation of social processes.

It is difficult to predict the effects that research and innovations in the “artificial intelligence” field could have. They might be considerable, but nothing can ensure that technical development itself would allow a society to arrive at a type of collective organization described by Iain M. Banks in his novels about the Culture. The uncertainties must not necessarily prevent a reflection on the potential consequences of possible leaps and bounds in computer developments. It is not only a technical question; its political implications must also be examined, especially how it could affect collective affairs. Technology and values are not two separate matters: values are part of technology, where some are favored and others disadvantaged. Fictional productions, by seizing technical potentialities in an imaginary form, may contribute to the initiation of forms of reflexivity and to contrasting different ways of implementing the technical promises. Iain M. Banks, when he describes this civilization that he calls the Culture, seems to postulate a form of equality between humans and machines, and, in any case, an absence of hierarchy. In this vast galactic universe, technological progress appears to have allowed the civilization to achieve a post-governmental regime in a society without class distinction. The capabilities of the “artificial intelligences” have eliminated any reason to entrust the management of collective affairs to a State or an administration. This model, which goes beyond the simple coexistence of entities of different natures, seems to postulate a general agreement regarding what the common good is. And the main divisions of interest appear to have disappeared. While the political model proposed by Iain M. Banks resembles a democracy, it is controlled and regulated by a new form of an elite whose judgement is taken for granted, an elite who would be of a particular technocratic form since it would be made up of “artificial intelligences”. Indeed, civilization would be going from one delegation to another. Moreover, it is interesting to examine the relationship between Banks’ model and theoretical reflections about democracy. Recent developments in these reflections have contributed to emphasizing the deliberative dimension of political activities, especially as a guarantee for the legitimacy of collective choices.55 In the works of Iain M. Banks, this deliberative dimension seems to be non-existent. Readers can only ask themselves if it is possible to debate with “artificial intelligences”, and a fortiori on a large scale. In what kind of public space would it be possible? An increasingly widespread presence of highly evolved machines leads to a profound questioning of the pertinence of decision-making. For human beings, how much individual autonomy remains? This type of collective functioning presupposes that trust can be put in machines (and in the new range of expert systems). However, it is not because progress in “artificial intelligences” would not produce a centralizing, hierarchical, and oppressive technology that it necessarily has liberatory potential.56 In the Culture novels, nothing is mentioned about how the “Minds’” activities can be supervised. That is, if the “Minds” let themselves be controlled… Far from being simplistic, the works of Iain M. Banks show the ambivalence of such a regime. Moreover, he associates the coming of a civilization like the Culture with the development of humanity in space: “In the purest sense, you get to the Culture almost whether you like it or not. But it does involve getting out to space, and it does involve just a huge amount of manufacturing capability. Because what you end up with is entities, space ships or whatever, that become self-sufficient and free moving in space, and it’s very hard to keep effective control of them”.57 If this step is the determining factor, it goes without saying that humanity still has a long way to go.