PolyWorld: Life in a New Context

Computational Genetics, Physiology, Metabolism, Neural Systems, Learning, Vision, and Behavior

or PolyWorld: Life in a New Context

Larry Yaeger
Apple Computer, Inc.
20525 Mariani Ave., MS 76-2H
Cupertino, CA 95014

1. Introduction

The study of living systems has taken many forms, from research into the fundamental physical processes to ethological studies of animal behavior on a global scale. Traditionally these investigations have focused exclusively on ÒrealÓ biological systems existing in our worldÕs ecological system. Only recently have investigations of living systems begun to occur in ÒartificialÓ systems in computers and robotic hardware.

The potential benefits of an enhanced understanding of living systems are tremendous. Some are of a grand scale, and are intuitively obvious, such as improvements in our ability to manage our own real ecosystems, the development of true machine intelligence, and the possibility of understanding our own mental and physiological processes. Some are of a more prosaic scale, but more accessible thereby, and, perhaps, of more immediate utility, such as simple learning systems, robust pattern classifiers, general purpose optimization schemes, robotic controllers, and evolvable software algorithms. The technological issues of the study of Artificial Life (ALife) are well laid out by Langton [27] in the proceedings of the first ALife workshop; the societal and philosophical implications of ALife are well presented in Farmer and Belin [16] in the proceedings of the second ALife workshop.

The ecological simulator, PolyWorld (PW), presented and discussed in this paper, is one instantiation of these ALife motivations and principles. PolyWorld attempts to bring together all the principle components of real living systems into a single artificial living system. PolyWorld does bring together biologically motivated genetics, simple simulated physiologies and metabolisms, Hebbian learning in arbitrary neural network architectures, a visual perceptive mechanism, and a suite of primitive behaviors in artificial organisms grounded in an ecology that is hopefully just complex enough to foster speciation and inter-species competition. Predation, mimicry, sexual reproduction, and even communication are all supported in a straightforward fashion. The resulting survival strategies, both individual and group, are purely emergent, as are the functionalities embodied in their neural network ÒbrainsÓ. Complex behaviors resulting from the simulated neural activity are unpredictable, and change as natural selection acts over multiple generations.

PolyWorld is a tool for investigating issues relevant to evolutionary biology, behavioral ecology, ethology, neural systems, and computer science. This paper will discuss the design principles employed in PW, along with some of the resulting behavior patterns observed in ÒspeciesÓ evolved in PW, their neural architectures, and the genetic variations observed in large populations under different ecological conditions.

2. Background

This work owes much in terms of inspiration to the work of W. Grey Walter [44,45,46], Valentino Braitenberg [3], Richard Dawkins [9,10,11], John Holland [21], Ralph Linsker [28,29,30], and John Pearson [36].

WalterÕs early work with simple electronic ÒturtleÓ nervous systems, and BraitenbergÕs ÒvehiclesÓ suggested the approach of utilizing simulated organisms. PolyWorld diverges from these works by encapsulating these organisms in a simulated world, and employing neural systems and learning rules from the world of computational neurophysiology, and by supporting a range of interactions between organisms. And though a number of other researchers (Travers [43]; Wharton and Koball [47]; and even a commercial product from Bascom software, the author of which is not known) have built simple Braitenberg vehicle simulators (or actual physical models in the case of Wharton and Koball), these typically concentrated on a wiring-diagram user interface, and implemented vehicles through only level number 2 (of 14). PW, on the other hand, takes note of the fact that by as early as Vehicle 4, Braitenberg invoked a form of natural selection, and supports the evolution of its organismsÕ Òwiring diagramsÓ, rather than having them specified by hand. The neural systems of PW also utilize Hebbian learning during the lifetime of an individual, which is undoubtedly purposefully similar to BraitenbergÕs Òmnemotrix wireÓ.

Richard DawkinsÕs writings communicate both the beauty and the effectiveness of evolutionary dynamics. In personal communications, he has also brought out key issues in speciation, such as the isolation of populations and the reduced viability of divergent species interbreeding, that have become important elements of this simulator.

The artificial neural systems employed in PW are based on Hebbian learning, and a novel approach to network architecture specification. Besides the obvious importance of Donald HebbÕs [18] research and speculations, their instantiation in the work of Ralph Linsker and John Pearson has guided the selection of these particular techniques for use in PW. LinskerÕs work demonstrated that Hebbian learning, as employed in PW, can and will self-organize important types of neural response patterns observed in early visual systems of real organisms. John Pearson, working with Gerald Edelman, utilized a variant of Hebbian learning and successfully demonstrated important principles of neuronal and synaptic self-organization Ñ cooperation and competition(for representing their observed inputs) Ñ that again correspond well to phenomena observed in real living systems. PolyWorld takes this unsupervised learning technique, and embeds it in arbitrary, evolving neural architectures, and then confronts the simulated neural system with survival tasks in a simulated ecology.

In the last couple of decades, a number of researchers have developed computational ecologies targeted at various scientific issues. Conrad [7,8] and Packard [34] have built systems to explore fundamental principles of evolutionary dynamics. Jefferson et al [23], and Collins and Jefferson [6] have constructed systems dealing with evolutionary mechanisms, behavioral goals, and learning architectures (Finite State Automata vs. Neural Networks). Taylor et al [40] developed a system to investigate the relationship between individual behavior and population dynamics. Ackley & Littman [1] built such a simulator to demonstrate a novel mechanism by which evolution can guide learning. Peter Todd and Geoffrey Miller [31,41,42] have explored evolutionary selection for different learning algorithms in organisms with simple vision systems and an innate sense of ÒsmellÓ that functions with varying degrees of accuracy. Danny Hillis [19,20] has used simple computational ecologies to evolve ÒrampsÓ, and exchange-sort algorithms. Core Wars [13,14,15] is a non-evolving ecology of code fragments, and RasmussenÕs VENUS [37] is an evolving system based largely on Core Wars. Thomas Ray [38] has also developed a computational ecology, TIERRA, based on evolving code fragments. And John Koza [26] has developed a system for evolving LISP functions that he terms ÒGenetic ProgrammingÓ. PolyWorld, in its original conception, was targeted principally at the evolution of neural architectures for systems faced with complex behavioral tasks; however, its biologically motivated behavioral and reproductive strategies, and the evolutionary mechanisms employed also make it suitable for use in behavioral ecology and evolutionary biology. The extent of PWÕs fidelity to biological systems, together with its unique use of a naturalistic visual perceptive system to ground its inhabitants in their environment distinguish it significantly from previous ecological simulators.

John HollandÕs ECHO system explicitly models a form of predation, involving ÒoffenseÓ and ÒdefenseÓ genes that determine the outcome of violent encounters. Holland notes that in his system, this form of predation was essential to the evolution of complex genomes. Though not as crucial to PWÕs genetic complexity, predation was also designed into PW from the beginning. In PW, genes also affect the outcome of violent encounters between organisms, but more indirectly through their ÒphysiologicalÓ characteristics (strength and size). There is also a behavioral component to the outcome of these encounters in PW, namely the degree of ÒvolitionÓ associated with the ÒfightingÓ behavior (the activation level of a predefined ÒfightÓ neuron), that differs from ECHOÕs handling of predation.

Belew et al [2] give an excellent overview of recent work in the area of evolving neural networks. Reviewed briefly there, and presented in detail in their own paper, Harp et al [17] have developed a scheme for evolving neural architectures that has an element of ontogenetic development. Their approach involves a set of synaptic projection radii between neuronal ÒareasÓ. PolyWorldÕs scheme for evolving architectures relies on the specification of connection densities and topological distortion of connections between neuronal groups. These architectural criteria are represented in the genome, and then expressed as an organismÕs neural architecture at ÒbirthÓ. This technique, though perhaps not quite as developmental as HarpÕs approach, or the non-neural, but very biologically motivated cellular growth work of de Boer et al [12], has the strengths of being much more developmental (and representationally compact) than a simple encoding of synaptic efficacy in the genes, and being computationally very efficient. It captures the statistical results of development, without the necessity of modeling the developmental process itself.

David Chalmers [4] has experimented with evolving supervised neural network learning algorithms, successfully evolving the classic "delta rule" for a linear, single layer perceptron, and speculated on applying this "genetic connectionism" approach to other architectures and learning algorithms. He also varies the diversity of his learning tasks, and demonstrates a correlation between this diversity and the generality of the evolved learning algorithm, similar to the correlation observed between amount of training data and generalization in supervised, "Back-Prop" neural networks. Though the evolution of unsupervised learning algorithms is one area of special interest to the author, the current version of PW has the classic "Hebb rule" built in. Neural architectures are, however, evolved in PW. Interestingly, by permitting free movement in a simulated environment, PW effectively can generate an unlimited amount of diverse input for the neural mechanisms employed by its denizens.

Nolfi et al [33] and Parisi et al [35] have explored evolving the connection strengths in small, fixed-architecture feed-forward neural networks controlling simple movement strategies in organisms evolved to seek food. The organisms are directly provided with angle and distance to food items, and are alone in their environment. Nolfi, Parisi, et al also introduce a "self-supervised" learning technique, using the traditional back-propagation of error algorithm, and demonstrate an improvement in evolved foraging efficiency associated with a learned ability to predict the sensory consequences of motor activity. PW employs an unsupervised learning algorithm and arbitrary neural architectures, with a more biologically- motivated vision mechanism, as well as a competitive ecology.

For the purpose of computer graphics animation, Renault et al [39] have experimented with visual systems for controlling computer generated characters. Their system goes beyond visual processing, however, to include unique object identification and distances to objects as part of the input to the character control programs. These control programs are rule-based and completely hand-crafted, specifically to provide obstacle avoidance. In contrast, PW uses only the pixel colors associated with visual processing, and provides these as input to the non-rule-based neural systems of evolving organisms, without specifying the meaning or use of this information.

Dave Cliff [5] has implemented a neural visual system for a simulated fly, and states that it is only by a grounding perceptive mechanism such as vision that neural models can be made sense of. For the purposes of his simulation, the model fly is attached to a rotating, but otherwise unmoving test-stand similar to real experimental setups. Organisms in PW use vision as their primary sense mechanism, but are free to explore their environment, and must do so effectively Ñ using their vision to guide a suite of primitive behaviors Ñ in order to survive and reproduce.

The study of real living systems has spanned many physical and temporal scales, from molecular level biochemical processes that take place in nanoseconds, through cellular level neural processes with time scales of a few milliseconds, to global evolutionary processes occurring over geological time scales. One of the first decisions necessary to commence an investigation into artificial living systems is that of scale: At what level of detail is it desirable to specify the parameters and underlying models of the simulation, and at what level does one wish to observe the resultant behaviors? Given current constraints on compute power, it is simply not feasible to begin computation with sub-atomic physics and expect to observe ethological behaviors. Since ecology-level dynamics were the desired output level of the system being designed, it was clear that behavior models for PWÕs individual organisms could not be too complex. However, a desire to avoid rule-based behavior specification led to a decision to model the organismsÕ behaviors at the neuronal level. Since even natural evolutionary forces are constrained by their previous successes, the real world has filled up with organisms exhibiting a wide range of variations on assemblages of neuronal cells (in addition to other cell types, of course). Modeling PWÕs organisms at this level permits us to sidestep millions of years of evolution, while still taking advantage of its results to date. In many ways, PW may be thought of as a sort of electronic primordial soup experiment, in the vein of Urey and MillerÕs [32] classic experiment, only commencing at a much higher level of organization, with light- sensitive and neuronal cells as the ingredients, and a simple ecology that includes all the other assemblages of those cells Ñ the other organisms in the world Ñ as the environment.

3. Overview

PolyWorld is an ecological simulator, of a simple flat world, possibly divided up by a few impassable barriers, and inhabited by a variety of organisms and freely growing ÒfoodÓ. The inhabiting organisms use vision as input to a neural network brain that employs Hebbian learning at its synapses. The outputs of this brain fully determine the organismsÕ behaviors. These organisms and all other visible constituents of the world are represented by simple polygonal shapes. Vision is provided by rendering an image of the world from each organismÕs point of view, and using the resulting pixel map as input to the organismÕs brain, as if it were light falling on a retina.

A small number of an organismÕs neurons are predetermined to activate a suite of possible primitive behaviors, including eating, mating, fighting, moving forward, turning, controlling their field of view, and controlling the brightness of a few of the polygons on their bodies. Organisms expend energy with each action, including neural activity. They must replenish this energy in order to survive. They may do so by eating the food that grows around the environment. When an organism dies, its carcass turns into food. Because one of the possible primitive behaviors is fighting, organisms can potentially damage other organisms. So they may also replenish their energies by killing and eating each other. Predation is thus modeled quite naturally.

The organismsÕ simulated physiologies and metabolic rates are determined from an underlying genome, as are their neural architectures. When two spatially overlapping organisms both express their mating behavior, reproduction occurs by taking the genetic material from the two haploid individuals, subjecting it to crossover and mutation, and then expressing the new genome as a child organism.

One way to look at this artificial world is as a somewhat complex energy balancing problem. The fittest organism will be the one that best learns to replenish its energies by eating, and to pass on its genes by mating. The particular patterns of activity that a successful organism engages in - the methods by which it sustains and reproduces itself - will be optimal for some particular fitness landscape. But since that fitness landscape depends upon the behavior of the world's other inhabitants, it must, per force, be a dynamic landscape. Since there is considerable variation in the placement and behavior of food and other organisms in the world, that fitness landscape is also fundamentally stochastic. Indeed, if the "fittest organism in the world" fails to find a suitable mate in order to pass on the important bits of its genetic material, then those genes will be lost... possibly for all time. Accordingly, every world has the potential to be quite different from every other world.

Once an Evolutionarily Stable Strategy (ESS) has emerged, there is no fitness function except survival. Until an ESS has emerged, PW is run in a sort of Òon-line Genetic Algorithm (GA)Ó mode, with an ad hoc fitness function. During this stage, a minimum number of organisms may be guaranteed to populate the world. If the number of deaths causes the number of organisms extant in the world to drop below this minimum, either another random organism may be created by the system, or the offspring of two organisms from a table of the N fittest may be created, or, rarely, the best organism ever may be returned to the world unchanged. This ad hoc fitness function rewards organisms for eating, mating, living their full lifespan, dying with reserve energies, and simply moving. Each reward category is normalized by the maximum possible reward in each category, and has a specifiable scale factor to permit easy tuning of the fitness function. Some simulation runs acquire an ESS in the first seed population and never require this on-line GA stage. Throughout this paper, the term created is applied to organisms spontaneously generated by the system, while born is used to refer to organisms resulting from the mating behaviors of the organisms.

Current high end simulations typically involve over 300 organisms, with up to approximately 200 neurons each, and require about 13 seconds per time-step on a Silicon Graphics Iris 4D/240- GTX. With an average lifespan of about 500 time-steps, and a time- to-first-offspring of about 100 time-steps, this means that 500 generations can be run at this complexity in about 1 week. More modest simulations with around 100 comparable organisms require about 4 seconds per frame, and take a day or two for the same task. And at the low complexity end, simple demonstration worlds can be run in Òreal timeÓ, at a few frames per second, and allow a more interactive experience for learning the system.

Figure 1 shows a sample view of the PolyWorld terrain, populated with three related but distinct sub-species. The largest panel shows a broad view of the world: the dark green ground plane, the brown, impassable barriers, the bright green pieces of food, and the multicolored organisms. Just above this oblique world view are four graphs of various informative simulation parameters. Above these at the top of the figure are many small views of the world drawn from the point of view of each of the organisms in the world; these are the images seen by the those organisms. At the top right are a few numerical statistics. And in the bottom right pane is a close-up view of the current ÒfittestÓ organism.

4. Genetics

An organismÕs genes completely encode both its ÒphysiologyÓ and its neural architecture. Table 1 lists the full complement of genes present in the organisms of PW.

¥ size ¥ strength ¥ maximum speed ¥ ID ¥ mutation rate ¥ number of crossover points ¥ lifespan ¥ fraction of energy to offspring ¥ number of neurons devoted to red component of vision ¥ number of neurons devoted to green component of vision ¥ number of neurons devoted to blue component of vision ¥ number of internal neuronal groups ¥ number of excitatory neurons in each internal neuronal group ¥ number of inhibitory neurons in each internal neuronal group ¥ initial bias of neurons in each non-input neuronal group ¥ bias learning rate for each non-input neuronal group ¥ connection density between all pairs of neuronal groups and neuron types ¥ topological distortion between all pairs of neuronal groups and neuron types ¥ learning rate between all pairs of neuronal groups and neuron types

Table 1. List of genes in organisms of PolyWorld.

All genes are 8 bits in length, and may be Gray-coded or binary-coded. All but the ID gene are used to provide 8 bits of precision between a specifiable minimum and maximum value for the corresponding attribute. For example, if the minimum possible size is minSize, and the maximum possible size is maxSize, and the value of the size gene (scaled by 255 to lie between 0.0 and 1.0) is valSizeGene, then the size of the organism with this gene will be:

size = minSize + valSizeGene * (maxSize - minSize)

These extrema values, along with a variety of other controlling parameters for the simulation, are contained in a Òworld fileÓ that is read by the simulator at startup.

The first 8 genes control the organismÕs simulated physiology. Its size and strength affect both the rate at which it expends energy and the outcome of ÒfightsÓ with other organisms. In addition, its size is related directly to the maximum energy that it can store internally. The next gene, maximum speed, also affects its ÒmetabolicÓ rate.

The ID geneÕs only function is to provide the green component of the organismÕs coloration at display time. Since organisms can actually see each other, this could, in principle, support mimicry. For example, a completely passive species could evolve to display the green coloration of a very aggressive species if it were of selective advantage. It might also be possible to attract potential mates by displaying the green coloration of food, though this might be of limited survival value. (In practice, however, neither of these somewhat sophisticated evolutionary responses has yet been observed.)

Mutation rate, the number of crossover points used during reproduction, and maximum lifespan were placed in the genes in order to permit a kind of meta-level genetics, and in recognition of the fact that these parameters were themselves evolved in natural systems. They are, however, typically constrained to operate within ÒreasonableÓ limits; 0.01 to 0.1 for mutation rate, 2 to 8 for number of crossover points, and a few hundred to a few thousand Òtime- stepsÓ for lifespan.

The final physiology gene controls the fraction of an organismÕs remaining energy that it will donate to its offspring upon birth. The offspringÕs total available energy on birth is the sum of these contributions from the two parents. Accordingly, at least one aspect of sexual reproduction may be captured by PWÕs evolutionary ÒbiologyÓ: it is entirely possible for two interbreeding sub-species to be almost identical genetically, differing only in the amount of personal energy devoted to the reproductive process. PolyWorld has not yet been instrumented to observe for this phenomenon.

The remaining genes are used to define the organismÕs neural architecture. These control parameters will be discussed in the section on Neurons and Learning. It should be noted here, however, that one of the motivations for this method of specifying the neural architecture was to reduce the number of genes necessary to specify the neural system. Early versions of PW used a simpler, fully recurrent neural architecture, and maintained a complete matrix of synaptic efficacies between all pairs of neurons in the genes. For 200 (NN) neurons, this older model required 40,000 (NN2) genes. The current scheme supports evolving neural architectures which are fully specified by 12NG2 + 232NG + 1026, where NG is the number of internal neuronal ÒgroupsÓ or clusters (output group sizes are fixed to 1, and input groups do not need biases, bias learning rates, or incoming synaptic connections). Thus, for 4 internal groups, with up to 32 neurons per group, plus up to 16 neurons per vision group (of which there are 3, one for each color component: red, green, blue), plus 2 other input groups (one neuron per group), plus the standard 7 output groups (one neuron each), a network of up to 185 neurons can be fully specified by just 2,146 genes. The large constants in this equation (232 and 1026) are due to the fixed set of input and output groups, and, especially, the desire to maintain each output neuron as a distinct group. Though the number of specifications are significantly reduced from a full crossbar matrix, this number still heavily outweighs the number of genes devoted to physiology. To permit a more robust exploration of the space of possible physiologies, then, the first crossover during genetic reproduction is always forced to occur somewhere within the set of physiology genes.

An organismÕs genome is allocated and interpreted such that space is available for the maximum possible number of neuronal groups. That is, one of the parameters specified per pair of groups in a network with 3 groups out of a maximum of 5 groups would be accessed as:

1,1 1,2 1,3 -- -- 2,1 2,2 2,3 -- -- -- 3,1 3,2 3,3 -- --

where the entries marked Ò--Ó serve simply as place holders. This is as opposed to an access scheme looking like:

1,1 1,2 1,3 2,1 2,2 2,3 3,1 3,2 3,3

where the entries are contiguous. The reason for this is to permit a smoother evolution of these neural architectures. The addition of a fourth group would leave the old connections intact in the first representation, but not in the second. It is even possible for a useful subcomponent of the architecture to ride along dormant in the genes to be expressed at a later time.

Though learning is supported in the neural network model employed in PW, only the architecture and some initial values are encoded in the genes; hence evolution in PW is purely Darwinian, not Lamarckian.

As in most GAÕs, when an organism is created from scratch, the bits in its genes are first zeroed, and then turned on with a certain bit probability. Unlike most GAÕs, it is possible to specify a range of legal bit probabilities, rather than always using 0.5. The bit probability for an individual organism is then randomly selected from this range and used to initialize the organismÕs bit-string genome. So the probability of a bit being on in a particular organism will depend on the value randomly selected from the specified range, while the probability of a bit being on in the population as a whole will just be the mean of the specified range (0.5 if the range is 0.0 to 1.0). This permits a wider variance in local, organism-specific bit probabilities in early populations, rather than depending entirely on mutation and cross-over to so shuffle the bits. Whether this is of any real value should be tested in a simpler GA system, and may be problem-specific in any event. Here it was felt that both the older fully-recurrent neural network architecture and the later evolving neural architectures were more likely to have behaviorally/evolutionarily useful solutions with lower bit densities; this provided a mechanism for so biasing the initial seed population without ruling out selection towards the unexpected end of the spectrum.

There is an optional Òmiscegenation functionÓ (so dubbed by Richard Dawkins), that may be used to probabilistically influence the likelihood of genetically dissimilar organisms producing viable offspring; the greater the dissimilarity, the lower the probability of their successfully reproducing. This function is not typically invoked until after a (specifiable) ÒsignificantÓ number of births without an intervening creation in order to allow the early stages of the simulation to explore as many genetic recombinations as possible.

5. Physiology and Metabolism

As discussed above, the simulated physiology of PolyWorldÕs organisms is determined by their genes. The size of the organism directly affects the maximum amount of energy that the organism can store. If an organismÕs size is allowed to range between minSize and maxSize, and its energy capacity ranges between minECap and maxECap, then a given organismÕs actual energy capacity, ECap is given as:

ECap = minECap + (size - minSize) * (maxECap - minECap) / (maxSize - minSize)

Similar linear relations are used to determine the influence of an organismÕs size on the rate at which it expends energy during forward or turning movement (relative to a specifiable maximum- size-penalty), and a size-advantage it will have during a fight with another organism (relative to a specifiable maximum-size- advantage).

An organismÕs strength also affects both its energy expenditure and its advantage in a fight. Strength directly scales the total energy used in a given time step, and thus usually ranges around 1.0 (typically 0.5 to 2.0). An attackerÕs strength also scales the effect on the victimÕs energy loss (fighting is discussed in more detail below in the section on Behavior).

The energy expended by an organismÕs neural processing is determined linearly from the number of neurons and the number of synapses it has. A maximum number of neurons and synapses is determined from the control parameters for the entire world, then each individualÕs neural energy expenditure is computed relative to these maxima. Globally applied Òneuron-to-energyÓ and Òsynapse-to- energyÓ conversion factors then multiply these scaled neuron and synapse counts to determine the actual energy expended per time step.

There are similar behavior-to-energy conversion factors for each of the primitive behaviors (eating, mating, fighting, moving, turning, focusing, and lighting). The total energy expended in a time step is then the activation (0. to 1.) of the corresponding output/behavior neuron multiplied by that behaviorÕs energy- conversion factor, summed over all behaviors, plus the neural energy expenditure, plus a specifiable fixed energy drain, with this sum finally scaled by the organismÕs strength.

As should be evident, there are clear energy conservation benefits to being small and weak, yet there are clear predatory advantages to being large and strong. Size also permits an overall greater capacity to store energy, thus making energy available for additional behavioral activity, including reproduction. The interplay between these opposing advantages is intended to produce niches in the fitness landscape, which may change over time. There are similar opposing pressures between energy expenditure and visual acuity on the number of input neurons devoted to vision.

There are two classes of energy storage in each organism: health-energy, and food-value-energy. Both are replenished by eating food. Both are depleted by neural activity and by engaging in the various behaviors. But when an organism is attacked, only its health-energy is depleted by the attack. If this health-energy reaches zero, the organism dies. When an organism dies it is converted into a piece of food containing an amount of energy equal to the organismÕs food-value-energy. This separation of health- energy from food-value-energy makes the predator-prey interactions quite natural; i.e., it is possible for an organism to be killed by having its health-energy driven to zero, while still maintaining a relatively high food value for the attacker.

An organismÕs food-value-energy will always be greater than or equal to its health-energy, yet both classes of energy have the same maximum capacity. Accordingly, an organism may continue to eat to replenish its health-energy after its food-value-energy has reached capacity. It is the health-energy that is provided as input to the neural network (see next section), and that is used to determine the amount of energy to be transferred to offspring.

Purely for the purposes of display, an organismÕs length and width are scaled by (the square root of) its maximum speed, length being multiplied, width being divided. Thus faster individuals will appear longer and sleeker, while slower individuals will appear shorter and bulkier. Since an organismÕs visual acuity is subject to evolutionary pressures, it is conceivable that an organism might emerge that was able to ascertain another organismÕs maximum speed purely from its shape, if there was a great enough advantage to the acquisition of this information (though this is not expected to be the case).

6. Neural Systems and Learning

The inputs to an organismÕs neural network ÒbrainÓ are its ÒvisionÓ, the current normalized level of its internal health-energy store, and a random value. The outputs are the suite of 7 possible primitive behaviors (eating, mating, fighting, moving, turning, focusing, and lighting). The internal neurons and all of the synaptic connections have no prespecified functionality; their utility is determined entirely by genetics and natural selection.

The form of an organismÕs brain, or neural system, is fully characterized by a set of parameters that are encoded in its genes. Referring back to Table 1, notice that the number of neurons devoted to each color component of vision is specified separately, permitting a specialization for more resolution in the most effective color, should this be of selective advantage. These numbers typically range between 1 and 16 neurons per color.

Next is a parameter that specifies the number of internal neuronal groups or clusters. This typically ranges from 1 to 5. In addition, there are 5 input groups (red vision, green vision, blue vision, energy level, and random), plus 7 output groups (the behaviors listed above).

Each neural group may have distinct populations of excitatory (e-) and inhibitory (i-) neurons. The number of e- and i-neurons are specified on a per group basis, and typically range between 1 and 16 neurons of each type. Synaptic connections from e-neurons are always excitatory (ranging from 0.0 to a specifiable maximum efficacy). Synaptic connections from i-neurons are always inhibitory (ranging from -1.e-10 to the negative of the maximum efficacy).

Though the bias on each of the non-input neurons varies during the simulation, the initial values for these biases and their learning rates are specified on a per group basis, for each of the non- input neural groups. Biases are updated by a Hebbian learning rule, as if it were a synaptic connection to a neuron that was always fully activated, but unlike other synapses in this network, the bias may change sign. Biases typically range from -1.0 to 1.0, and bias learning rates typically range from 0.0 to 0.2.

The remaining parameters - connection density (CD) , topological distortion (TD), and learning rate (LR) - are all specified for each pair of neuronal groups and neuron types. That is, separate values for each of these parameters are specified for the excitatory- to-excitatory (e-e), excitatory-to-inhibitory (e-i), inhibitory-to- inhibitory (i-i), and inhibitory-to-excitatory (i-e) synaptic connections between group i and group j, for each pair of groups i and j.

Connection density, as the name suggests, is used to determine the extent of the connectivity between neuronal groups. The number of e-e synapses between group i and group j is given by the nearest integer to CDe-e(i,j) * Ne(i) * Ne(j), where CDe-e(i,j) is the e-e CD from group j to group i, Ne(i) is the number of e-neurons in group i, and Ne(j) is the number of e-neurons in group j. Similar expressions hold for the other types of connections between all pairs of groups. CD can range from 0.0 to 1.0.

Topological distortion is used to determine the degree of disorder in the mapping of synaptic connections from one group to the next. That is, for a TD of 0.0, synapses are mapped to perfectly contiguous stretches of neurons in the adjacent layer; for a TD of 1.0, synapses are mapped in a completely random fashion between adjacent layers. Thus retinatopic maps such as are observed in natural organisms can be enforced (or not) at the architectural level (as well as resulting from the learning process). TD typically ranges from 0.0 to 1.0.

Learning rate controls the Hebbian learning process at the synapses between each pair of neuronal groups. This permits natural selection to favor hardwired, ÒinstinctiveÓ connections for some neural pathways, while supporting learning in other pathways. LR typically ranges from 0.0 to 0.2.

This method of specifying the neural architecture is fairly general, and is not biased for any particular neural organization. Possibly, one might expect to evolve a preponderance of inhibitory connections, especially locally, if the simulated neural architectures evolve to match real neural systems; yet the possibility exists for establishing local excitatory connections (such as are found in CA3 in the hippocampus). The technique does not, however, explicitly model architectures whose characteristics are heavily based upon spatial organization (such as the parallel fibers originating from the granule cells in the cerebellum). A straightforward extension to the current method, that allowed unique specifications of the same parameters along multiple spatial dimensions, could account for such organizational schemes. However, with the limited compute resources currently being applied to PW simulations, and thus the limited number of neurons permitted in each brain, it was not deemed worthwhile to further decompose the groups into these spatial subcategories.

When an organismÕs brain is ÒgrownÓ from its underlying genome, the synaptic efficacies are randomly distributed between specifiable minimum and maximum values. The brain is then exposed to a sequence of mock visual inputs consisting of random noise, for a specifiable number of cycles. This is all pre-birth. In this fashion, it is unnecessary to store any synaptic efficacies in the genes. This approach was inspired by LinskerÕs simulations of visual cortex, which gave rise to on-center-off-surround cells, orientation- selective cells, and so on, when exposed only to noise. The crucial aspects of the networks in this case are their architecture Ñ layered receptive fields in LinskerÕs case, evolved arbitrary topology in PW Ñ and the learning rule Ñ Hebbian learning in both cases.

It was debated whether to update all the organismsÕ brains synchronously or not. That is, whether each organismÕs neural network should be allowed to make a complete neural activation and synaptic learning pass with each time step. Even though it was desired to penalize organisms that evolved additional neurons and synapses, synchronous updating was ultimately selected, primarily because the corresponding structures in nature are executed in parallel, and penalties based on their serial implementation would be excessive. The penalty is more properly derived from the additional energy use associated with these additional neural structures.

At each time step, the input neurons are set to the appropriate values, corresponding to the organismÕs visual field, its current health-energy level, and a random number. New neuronal activations are computed by the simple formulae:

xi = S ajt sijt

ait+1 = 1 / (1 + e-axi)

where ajt is the neuronal activation of neuron j at time t (the beginning of this time step), sijt is the synaptic efficacy from neuron j to neuron i at time t, ait+1 is the neuronal activation of neuron i at time t+1 (the end of this time step), and a is a specifiable logistic slope.

The synaptic efficacies are then updated according to a Hebb rule, as:

sijt+1 = sijt + hckl (ait+1 - 0.5) (ajt - 0.5)

where sijt+1 is the synaptic efficacy from neuron j to neuron i at time t+1, and hckl is the learning rate for connections of type c (e-e, e-i, i- i, or i-e) from group l to group k. An optional multiplicative decay term may also be applied to the synaptic efficacy.

This simple Òsumming and squashingÓ neuron and Hebbian update rule are certainly coarse abstractions of the complexities observed in real neural systems. Credence is lent to these particular abstractions by the previously quoted simulation work of Linsker, Pearson, and others, and by LinskerÕs and othersÕ information- theoretic analytical work on such systems, which suggest that they may capture the information-processing attributes of real neural systems, if not their precise method of action. These neuronal and learning models were selected for use in PW based on these results and the modelsÕ computational tractability.

During the course of a simulation, neural and synaptic activities may be monitored for a number of organisms in the world (the top five ÒfittestÓ, according to the ad hoc fitness function discussed earlier, even if it is not being used to create new organisms). An example is shown in Figure 2. Gray-scale is used to denote neural activation (between 0.0 and 1.0) and synaptic efficacy (between -maxEfficacy and +maxEfficacy). At the very bottom of the grid is the color vision buffer. The neural activations at the beginning of this time step are shown in a horizontal row just above the color vision, along with the red, green, and blue input neuron activation levels, and the energy and random input neuron activation levels. White frames are drawn around each neuronal group, except for the vision neurons which are framed in their corresponding color. Black frames are drawn around each synapse, hence the unframed areas are regions of null connectivity. Synapses that appear brighter than the neutral gray background are excitatory; those that appear darker than the background are inhibitory. The leftmost vertical bar shows neuronal biases. The non-input neural activations at the end of this time step are shown in the adjacent vertical bar; again, neuronal groups are framed in white. Hence the diagram may be read as an incomplete crossbar connecting the neuronal states at the beginning of the time step (horizontally) to those at the end of the time step (vertically), through the various synaptic connections.

Early simulations with PW had much simpler, fully recurrent neural architectures. Though not particularly representative of real biological neural architectures, acceptable behavior strategies were evolved, and some of the results being presented are from organisms using these early networks.

7. Vision

The color vision supplied as input to the organism is first rendered at the minimum window size permitted on the Iris + 1 (because even-sized buffers can be accessed faster), or 22 x 22 pixels. The pixel row just above vertical center is then properly anti- aliased into whatever number of visual neurons an organism has. Even though organisms and the environment of PW are three- dimensional, the organismsÕ vision consists of just this one- dimensional strip of pixels, rather than the complete pixel map. Since the organisms are confined to motion on a ground-plane, it was felt that the benefit derived from computational efficiency outweighed the small loss of information resulting from this restriction.

As was discussed above in the Genetics section, the number of neurons devoted to each of the color components is evolved independently (though they are adjacent on the genome, and so may tend to crossover together).

As was discussed in the Neurons and Learning section, an organismÕs vision is shown in the display of the brain internals that may be invoked interactively for some of the ÒfittestÓ individuals. In addition, the full 22 x 22 pixel map for each of the organisms is usually displayed at the top of the screen. This is mostly for a Òreality checkÓ Ñ visual reassurance that the organisms are seeing what they would be expected to see, and may be disabled for a slight speed gain.

The vertical field of view of the organisms is fixed at 10o, since they only see a strip of pixels just above the center of the image. Their horizontal field of view, however, is under their own ÒvolitionalÓ, neural control. That is, the activation of the focusing neuron is mapped between a minimum and maximum field of view (typically 20o to 120o). In principle, this might permit some depth of field determinations based on cyclic focusing operations, though nothing so sophisticated has emerged (or is expected to emerge) in the limited neural systems employed by the organisms so far.

This type of direct perception of the environment should answer one of cognitive psychologyÕs most frequently sounded complaints against traditional AI: The organisms of PW are ÒgroundedÓ in their environment by their sense of vision.

8. Behavior

A suite of primitive behaviors is made available to all organisms in PW, namely:

¥ eating ¥ mating ¥ fighting ¥ moving ¥ turning ¥ focusing ¥ lighting

All of these behaviors are expressed by raising the activation level of a prespecified neuron in the brain. Given computational constraints, it was felt that a minimum number of cycles should be devoted to motor activity, hence this simple one-neuron-one- behavior mapping. The first three behaviors, eating, mating, and fighting, all have some associated threshold that must be exceeded before the activity is initiated. Energy is expended by each of the behaviors, including eating. The energy expenditure rates are controllable by scale factors (see Physiology and Metabolism) in the Òworld fileÓ (see The New Context).

Eating is an organismÕs method for replenishing depleted energy stores. In order to eat, an organismÕs position must cause it to overlap a piece of food. The amount of energy consumed is proportional to the activation of the eating neuron, once that activation exceeds a specifiable threshold.

Mating is an organismÕs method for reproducing. In order to reproduce, an organismÕs position must cause it to overlap another organism, and both organisms must express their mating behavior in excess of a specifiable threshold. The outcome of the reproductive attempt may be affected by the miscegenation function (see Genetics), or by the maximum number of organisms permitted in the world (see The New Context). The organismÕs ÒdesireÓ to mate (the activation level of its mating neuron) is mapped onto its blue color component for display purposes; this coloration is visible to other organisms as well as to human observers.

Fighting is an organismÕs method for attacking another organism. In order to successfully attack the other organism, the attackerÕs position must cause it to overlap the attackee. Only one organism need express its fighting behavior to successfully attack another. The energy that is depleted from the prey is a function of the volitional degree of the attack (the activation of the predatorÕs fight neuron), the predatorÕs current health-energy level, the predatorÕs strength, and the predatorÕs size. The product of these contributing factors from the predator is scaled by a global attack-to- energy conversion factor to make the final determination of amount of energy depletion applied to the prey. If both organisms are expressing their fight behavior, the same computation is carried out reversing the roles of predator and prey. Each organismÕs desire to fight is mapped onto its red color component for display purposes; this coloration is visible to other organisms as well as to human observers.

Moving refers to an organismÕs forward motion. Unless an organism encounters a barrier, or the edge of the world, it will move forward by an amount proportional to the activation of its moving neuron.

Turning refers to a change in an organismÕs orientation on the ground-plane (yaw). An organism will turn about its y-axis by an amount proportional to the activation of its turning neuron.

Focusing refers to an organismÕs control over its horizontal field of view. As discussed in the Vision section, the activation of an organismÕs focusing neuron will be linearly mapped onto a range of possible angles to provide its horizontal field of view. This makes it possible for an organism to use its vision to survey most of the world in front of it or to focus closely on smaller regions of the world.

Lighting refers to an organismÕs control over the brightness of a cap of several polygons on the front face of its ÒbodyÓ. The activation of an organismÕs lighting neuron is linearly mapped onto the full 0 to 255 brightness range in all color components of these front polygons. Accordingly, a simple form of visual communication is possible, in principle, for the organisms inhabiting PW. (No evidence of their use of this form of communication has yet been found nor sought to date, though evidence of the organismsÕ use of vision for controlling locomotion has been observed.)

9. The New Context

The ÒworldÓ of PolyWorld is a flat ground-plane, possibly divided up by a few impassable barriers, filled with randomly grown pieces of food, and inhabited by the organisms previously described.

The number of organisms in the world is controllable by several means. First, a maximum number of organisms is specifiable, in order to keep the problem computationally tractable. Second, a minimum number of organisms is specifiable to keep the world populated during the early on-line GA stage (see Genetics). Finally, an initial number of organisms is specifiable to determine how many individuals to seed the world with at the start of the simulation.

Food is grown at a specifiable rate up to a specifiable maximum number of grown food items. The number of food items may be guaranteed to be kept between a specifiable minimum and maximum food count. Subject to this maximum, food is also generated as the result of an organismÕs death. The amount of energy in a piece of food that is grown is randomly determined between a specifiable minimum and maximum food energy. The amount of energy in a piece of food resulting from the death of an organism is that organismÕs food-value-energy (see Physiology and Metabolism) at death, or a specifiable minimum-food-energy-at-death.

An arbitrary number of barriers may be placed in the world, which inhibit movement of the organisms. These can serve to partially or completely isolate populations of organisms, and as such can contribute significantly to speciation (genetic diversity). For reasons of computational efficiency, they are typically placed parallel to the z (depth) axis, though this is not strictly necessary.

It is possible to manage the minimum, maximum, and initial numbers of organisms and food items, along with the ad hoc fitness statistics, simultaneously for a number of different independent ÒdomainsÓ. These domains must be aligned parallel to the z (depth) axis, and typically, though not necessarily, coincide with the divisions imposed on the world by the barriers. This permits the simultaneous ÒculturingÓ of completely independent populations when barriers extend the full length of the world, or limits the spread of genes between domains to those resulting from actual movement of organisms when the barriers are arranged so as to leave gaps for organisms to travel through. If the domain fitness statistics were not kept separately, then genes from one domain could migrate to another domain by virtue of their global fitness during the start-up on-line GA phase.

It is possible to set a flag such that the edges of the world act as barriers (the usual), wrap around, or arenÕt there at all. In this last case, PWÕs ground-plane acts much like BraitenbergÕs table top, with organisms that move past the edge of the world dying instantly.

Various monitoring and graphing tools exist to assist in following the progress of a simulation and in developing an understanding of the evolutionary and neural dynamics at work. As was mentioned earlier (in the section on Neural Systems and Learning), a display of the internal workings of any of the five ÒfittestÓ organisms may be called up at any time. In addition, a small window that maintains an overhead view of the world will automatically track that same organism upon request. This overhead window may also be zoomed in and out to follow the organism more closely.

Also available are graphic displays of the time histories of certain quantities of interest, including: (1) population sizes (overall and per domain), (2) the past maximum, current maximum, and current average values of the ad hoc fitness function, (3) the ratio of the number of organisms ÒbornÓ (by mating) to the sum of the number of organisms born and created, and (4) the ratio of the difference of food-energy in and food-energy out to the sum of these two values. These last two items in particular are important gauges of the course of the simulation. Item (3) will start at 0.0 and asymptote to 1.0 for successful simulations, in which at least one species has emerged with an ESS; it will peak well below 1.0 for unsuccessful simulations. Item (4) ranges from -1.0 to 1.0, and should asymptote to 0.0, for a world where energy is conserved. Three values are actually plotted for item (4): (a) the total food- energy, including the initial seeding of the world, which starts at 1.0 and should asymptote to 0.0, (b) the average food-energy, excluding the initial seeding of the world, which starts at 0.0, and rapidly becomes negative, but should also asymptote to 0.0, and © the current food-energy on a time-step by time-step basis, which fluctuates rapidly, but should cluster around the average food- energy.

One additional display can graphically present the results of an analysis of the genetic variability in the population. All pairs of organisms are examined to determine the magnitude of the Hamming distance between them in gene space, and a gray-scale plot is used to display normalized genetic distances for the entire population at each time step.

All of the simulation control parameters and display options are defined in a single Òworld fileÓ that is read at the start of the simulation. In addition, some of the display options can be invoked interactively at runtime.

There isnÕt space to go into many details of the code itself. However, it may be worth noting that it consists of about 15,000 lines of C++, and is entirely object oriented, except for a single routine devoted to handling the organism-organism and organism- food interactions (for reasons of computational efficiency). The organisms, food, and barriers are maintained in doubly-linked lists sorted on a single dimension (x). This simple data structure has minimal maintenance overhead, yet rules out most non-intersections very well, and permits a sorting algorithm to be used that capitalizes on the expected frame-to-frame coherency of organism positions. It runs on a Silicon Graphics Iris (to take advantage of its hardware renderer for all the vision processing), and uses a set of object oriented C++ graphics routines (included in the line count above) that wrap around the standard Iris graphics library.

10. Results: Speciation and Complex Emergent Behaviors

Despite the variability inherent in different worlds, certain recurring ÒspeciesÓ have occurred in a number of the simulations run to date. By ÒspeciesÓ, I mean groups of organisms carrying out a common individual behavior that results in distinctive group behaviors. Since the selection of these behaviors are derived from the activity of their neural network brains, and the success of these behaviors is partially a function of their physiologies, both of which are in turn based on the genome of the organism, the behavioral differences may generally be traced to the organismÕs genetic code. Hence these behavioral differences are representative of different genetic species.

A simulation is considered ÒsuccessfulÓ if and only if some number of species emerge which are capable of sustaining their numbers through their mating behaviors, and thus organism creations cease. These species can be said to have developed an ESS within the ecological environment of PW. The observational reports below only refer to ÒsuccessfulÓ simulations.

The first of these species has been referred to as the "frenetic joggers". In an early simulation without barriers, without a miscegenation function, and with borders that wrap around (essentially forming a torus), a population emerged that basically just ran straight ahead at full speed, always wanting to mate and always wanting to eat. That particular world happened to be benign enough, that it turned out they would run into pieces of food or each other often enough to sustain themselves and to reproduce. It was an adequate, if not particularly interesting solution for that world. And without the miscegenation function or any physical isolation due to barriers, whatever diversity was present in the early world population was quickly redistributed and blended into a single species that completely dominated the world for as long as the simulation was run.

The second recurring species has been referred to as the Òindolent cannibalsÓ. These organisms "solve" the world energy and reproduction problem by turning the world into an almost zero- dimensional point. That is, they never travel very far from either their parents or their offspring. These organisms mate with each other, fight with each other, kill each other, and eat each other when they die. They were most prevalent in simulations run before the parents were required to transfer their own energies to the offspring; the organisms of these worlds were exploiting an essentially free energy source. With proper energy balancing, this behavior was reduced to only an occasional flare-up near corners of the world, where some organisms with limited motor skills naturally end up congregating, sometimes for quite extended periods of time. It turns out that the primary evolutionary benefit associated with this behavior was the ready availability of mates, rather than the ÒcannibalisticÓ food supply. This was determined by completely eliminating the food normally left behind by an organismÕs death, yet still observing the emergence of such species. Large colonies of these indolent cannibals look from above like a continuous (non-gridded) version of ConwayÕs game of LIFE.

The third recurring species has been referred to as the Òedge runnersÓ. These organisms take the next step up from the cannibals, and essentially reduce their world to an approximately one- dimensional curve. They mostly just run around and around the edge of the world (which they are forcibly prevented from running off of in most of the simulations). This turns out to be a fairly good strategy, since, if enough other organisms are doing it, then some will have died along the path, ensuring adequate supplies of food. And mates are easily found by simply running a little faster or a little slower, running in the opposite direction, or simply stopping at some point and waiting for other runners to arrive (all of which behaviors have been observed). A form of this behavior persists even when barriers block access to the rest of the world; organisms still sometimes congregate along any edges, including the barriers. It has been suggested [22] that this may be a form of behavioral isolation, permitting this species to retain its genetic identity to the exclusion of other species.

Another species recently emerged as the first evolutionarily stable solution to a Òtable topÓ world Ñ one with no edges. These ÒdervishesÓ evolved a simple rapid-turning strategy that kept them away from the dangerous edges of the world, and yet explored enough of the world to bring them into contact with food and each other. While this basic behavioral strategy persisted for many hundreds of generations, evolution continued to explore optimum degrees of predation, in a sort of continuous prisonerÕs dilemma over optimum degrees of cooperation. Waves of varying levels of expression of the fighting behavior could be observed sweeping through several

distinct populations over evolutionary time-scales, with the greatest variation in behaviors clearly seen at the boundaries between these populations.

The most interesting species and individuals are not so easily classified. In some worlds where a single species has become dominant, the individualsÕ behaviors have still been quite varied. And in many worlds, no single species becomes obviously dominant. It is especially in these simulations that a number of complex, emergent behaviors have been observed, including: 1) responding to visual stimuli by speeding up, 2) responding to an attack by running away (speeding up), 3) responding to an attack by fighting back, 4) grazing (slowing upon encountering each food patch), 5) expressing an attraction to food (seeking out and circling food), and 6) following other organisms.

The first item is important in that it implies that conditions have been found that will cause evolution to select for the use of the organismsÕ vision systems. All four of the earlier, simpler speciesÕ behaviors would be appropriate even if these vision systems did not exist. Yet PW was built on the assumption that vision would be a powerful, useful sense mechanism that evolution could not fail to employ. Even a simple speeding up in response to visual stimulation could result in reaching food or a potential mate more effectively, and this was the first observed visual response to emerge.

The second and third items both represent reasonable responses to attack by a predator. Fleeing may reduce the effect of the attack, and fighting back is an energy-efficient use of the organismÕs own ability to fight (as opposed to expressing the fight behavior continuously which would expend unnecessary energy).

Strategies four and five represent efficient feeding strategies. As simple a survival skill as grazing might seem - to simply notice when oneÕs internal energy is going up, and cease moving until it stops going up - it was not observed until a fairly recent simulation. It is still not a wide-spread phenomenon, though a few instances have now been observed. Only the most recent simulation, as of this writing, has given rise to a population of organisms that seem to be able to actively seek out food and ÒorbitÓ it while eating; such ÒforagingÓ is clearly a valuable survival trait. These organisms appear to be drawn to the food as if there were a magnet or some point attractor located in the food and controlling the organismsÕ behavior, though no such mechanism exists in PW. Their attraction to the food is purely a result of selection forces acting on the the neural architecture connecting their vision systems to their motor systems.

The final, ÒfollowingÓ strategy has also emerged only in this most recent simulation. Clearly of value, whether for seeking a prey or a mate, this represents the most complex coupling of the vision sense mechanism to the organismsÕ motor controls yet observed. Small ÒswarmsÓ of organisms, and one example of a few organisms ÒchasingÓ each other were even suggestive of simple ÒflockingÓ behaviors.

All of these behaviors, being inherently temporal phenomena, require some sort of temporal medium for display. Short video clips of most of the above species and behaviors should be available in a companion videotape released by the publisher of this book.

11. Concluding Remarks

Real benefits have already begun to accrue from the studies of artificial neural systems. Meanwhile, the study of artificial evolution Ñ genetic algorithms Ñ is yielding insights into problems of optimization, and into the dynamics of natural selection. One form of the study of Artificial Life is the perhaps obvious combination of these two fields of research. Adding computer graphics visualization techniques yields the basic substrate of PolyWorld.

One of the primary goals set out for PW has already been met: the evolution of complex emergent behaviors from only the simple suite of primitive behaviors built into the organisms of PW, their sense mechanisms, and the action of natural selection on their neural systems. These recognizable behavioral strategies from real living organisms, such as ÒfleeingÓ, Òfighting backÓ, ÒgrazingÓ, ÒforagingÓ, ÒfollowingÓ, and ÒflockingÓ, are purely emergent in the PW environment. And built as they are from simple, known primitive behaviors, in response to simple, understandable ecological pressures, they may be able to remove a little bit of the mystery, if not the wonder, at the evolution of such behaviors in natural organisms. Indeed, the organisms of PW have evolved these higher- order behaviors by reproducing the same bottom-up complexity ordering of behavioral dynamics as are postulated to drive such phenomena in natural organisms: core motor controls, followed by approach and avoidance behaviors in individuals, followed by group flocking behaviors that emerge naturally from the individual approach/avoidance behaviors.

The simple but effective strategies evolved by organisms in the earlier, simpler simulations may be valuable as sort of Ònull hypothesesÓ about certain forms of animal behavior. In particular, aggregation and wall-following amongst these simple organisms occurs without need for elaborate behavioral strategies. It is sufficient that corners and walls obstruct simpler trajectories. Yet if enough organisms occupy these locational niches, it becomes a behavioral niche as well, by providing readily available mates, and an easily achieved form of behavioral isolation.

It is, perhaps, easier to contemplate and understand these behaviors in the simulated organisms of PW than it is in natural organisms, precisely because they are simulated. The blessing and the curse of Artificial Life is that it is much more difficult for humans to anthropomorphize (zoomorphize? biomorphize?) these organisms in a machine than it is natural organisms. This frees us from prejudices and preconceptions when observing and analyzing the behaviors of artificial organisms, yet the most highly motivated of ALife researchers is going to find it difficult to look at an artificial organism and declare it unequivocally alive.

As more and more sophisticated computational models of living systems are developed, it will be only natural to ask whether they are in fact really alive. To answer this, however, requires a resolution to probably the greatest unanswered question posed and addressed by the study of Artificial Life: ÒWhat is life?Ó Farmer & Belin offer an analogous question for consideration: ÒIf we voyage to another planet, how will we know whether or not life is present?Ó One might also ask: If we ÒvoyageÓ to an artificial world, how will we know whether or not life is present? Rather than just ignore this question, letÕs look briefly at the how the organisms in PolyWorld stack up against Farmer & BelinÕs list of Òproperties that we associate with lifeÓ (slightly abridged here for brevity):

¥ ÒLife is a pattern in spacetime, rather than a specific material object.Ó PW organisms are indeed patterns in a computer, rather than any specific material; they neither extend nor violate this first condition. ¥ ÒSelf-reproduction.Ó PW organisms certainly reproduce within the context of their world. ¥ ÒInformation storage of a self-representation.Ó PW organisms use an analog of the same storage mechanism Farmer & Belin mention for natural organisms: their genetic representation. ¥ ÒA metabolism.Ó A PW organismÕs metabolism effectively converts food found in the environment into the energy it needs to carry out its internal processes and behavioral activities, just as is the case in natural organisms. The metabolism in PW organisms is much simpler, but if the function is the same, does the complexity of the underlying process matter? ¥ ÒFunctional interactions with the environment.Ó PW organisms interact with their environment, including other organisms; the more sophisticated ones respond behaviorally to changes in the environment; such responses are purely under the control of the organism. ¥ ÒInterdependence of parts.Ó Following Farmer & BerlinÕs reasoning, PW organisms can and would die were they somehow separated from their internal energy store. And severing an organismÕs brain in two would not produce two organisms with behavior anything like the original. Unnecessarily (and perhaps inappropriately) stepping outside the bounds of the simulation, they would also die if their various procedures and data were destroyed or isolated. In either case, half an organism is no longer that organism, if it is any organism at all. ¥ ÒStability under perturbations.Ó PW organisms can survive small changes to their environment. Indeed, whole species have reemerged in entirely different simulations. Again stepping outside the simulation, whole species have emerged with and without any of a variety of errors in the code. ¥ ÒThe ability to evolve.Ó PW organisms clearly can and do evolve. There are undoubtedly limits to their evolution; e.g., they could not possibly evolve a sense of smell without programmer intervention. However, all natural organisms we know of have limits to their evolutionary capabilities: It is highly unlikely that humans could evolve a steel appendage; if Einstein is correct, it is absolutely impossible for them to evolve a method of personal locomotion that would exceed the speed of light. All organisms, natural or artificial, are bound by the physics of their universe. Similar to the question about metabolism, does the complexity of the underlying physics matter?

Somewhat surprisingly, then, it would seem that we either need to further refine our constraints on the definition of life, or welcome a new genus to the world.

Ultimately, the resolution to this question of life in artificial organisms is probably going to have to be based on a consensus, as with TuringÕs famous test for artificial intelligence. Perhaps in this case, however, the consensus of a knowledgeable and informed jury is needed, rather than that of TuringÕs unspecified, presumably average group of individuals. As with the debate about the ÒalivenessÓ of natural viruses being properly resident with biologists, the question of ÒalivenessÓ in artificial organisms is probably best argued by a combination of computer-aware biologists and biology- aware computer scientists.

12. Future Directions

The various species and behaviors that have emerged in the different simulations suggest that PW may be a rich enough simulation environment to pursue further evolutionary studies. In particular, a way of sort of ÒbenchmarkingÓ PW Ñ the way one compares the results of a computational fluid dynamics code to flow over a flat plate or a cylinder, or over an airfoil measured in a windtunnel Ñ may have presented itself in the form of optimal foraging strategies as studied in the field of behavioral ecology. A simple, canonical foraging experiment has been defined and analyzed, and is now being simulated with PW. Agreement or disagreement with the analytical model should be examined and understood.

The neural architectures that provide the most useful survival strategies should be analyzed and understood. It would also be fairly straightforward to encode an entire range of learning algorithms in the genes of the organisms in PW, and attempt to evolve the most effective learning algorithm, rather than assuming it to be Hebbian. (Some consideration has even been given the possibility of having the fundamental genetic representation of information Ñ the genetic code Ñ evolve.) At least it might worthwhile implementing cluster-to-cluster initial connection strengths, initial connection strength variances, and maximum connection strengths, to begin to hint at distinct cell types. Or it may be more worthwhile to jump directly to a more sophisticated cell model, capable of capturing the actual temporal dynamics of spike trains rather than average firing rates.

More environmental interactions should be supported, including the ability for the organisms to pick up, carry, and drop pieces of food, and perhaps even pieces of barrier material. This should yield useful reasons for organisms to cooperate, other than simply to reproduce.

Though not discussed in the earlier parts of the paper, energetics of the system have been observed to be crucial to the evolution of successful survival strategies. Mimicing the differences between energy-rich tropical zones and energy-starved polar zones in our one known, natural ecosystem, artificial life flourishes in energy-rich simulations, and languishes in energy-starved simulations. Perhaps someday it may be possible to make useful predictions about viable ranges of energy flux for natural systems from artificial ecologies like PW.

A quantitative assessment of the degree to which the isolation of populations affects speciation may be possible with PW. Some tentative first steps have already been taken in this direction, though questions remain about the most appropriate comparisons to make and the appropriate times at which to make these comparisons. This coupled with the problems associated with assuring the emergence of an ESS in every population, and the simple magnitude of processing time required to perform the simulations has delayed a complete series of experiments of this nature.

There are thousands of other interesting experiments that one might perform with this system, including: Monitoring brain size in otherwise stable populations, such as the "dervishes"... are smaller and smaller nervous systems actually being selected for? Monitoring the frequency and magnitude of attacks on other organisms as a function of their genetic (dis)similarity. Monitoring the amount of energy given to offspring in a single species... is there any indication of an asymmetric split into different relative contributions? Hand- tailoring a good neural architecture or two and seeding the world with these engineered organisms. Providing multiple internal, neural time-cycles per external, action time-cycle. Evolving three completely independent domains of organisms, with barriers in place, and then removing the barriers to observer the interspecies dynamics. It may even be possible to model the entire population of Orca whales that frequent the waters around Vancouver, and look for an evolutionary split into pods that travel little and eat essentially stationary food sources versus pods that travel widely and feed on fish, a very mobile food source. And on and on. In hopes that others may find PolyWorld to be a useful tool for exploring these kinds of questions, it has been made available via ftp from ftp.apple.com (130.43.2.3) in /pub/polyworld. Complete source code and some sample "world files" are provided.

In a more fanciful, and perhaps more visionary vein, it is hoped that, someday, one of the organisms in PolyWorld that demonstrates all the survival behaviors observed to date, plus a few others, could be transferred from its original environment to, say, a maze world, and become the subject of some classical conditioning experiments. KlopfÕs [24,25] success at demonstrating over 15 classical conditioning phenomena in a single neuron using differential Hebbian learning (he called it Òdrive-reinforcementÓ learning), strongly suggests that such phenomena should be demonstrable in PolyWorldÕs organisms.

And then, of course, there is simply Òmore, bigger, and longerÓ: More organisms, with bigger neural systems, evolving longer. As a gedanken experiment, consider just how much Òmore, bigger, and longerÓ might be useful: The current 102 organisms, 102 neurons, and 102 generations (approximately), could be expanded to 106 organisms, neurons, and generations, through an increase in compute power of about 1012. (Though this sounds like a tremendous increase to ask for, consider that the current simulation is running on a single, scalar workstation processor, not a vectorized, massively parallel processor, then extend todayÕs trends in compute power, and this ceases to be such a daunting request; in fact, the compute power may be significantly less than this due to the greatly reduced motor and autonomic nervous systems that would be required by artificial organisms.) It turns out that this is a fairly reasonable amount of compute power with which to consider modeling a complete human brain Ñ basically devoting one of todayÕs fast computers to every neuron Ñ but no one understands how to actually construct such an artificial brain. However, this same amount of compute power might be used to evolve the equivalent of a new species of computational lab rat every week... and this is how: by combining evolution, neural systems, and ecological dynamics. The leap of (informed) faith is this: If it is actually possible to evolve a computational lab rat Ñ and the experiments so far in PW suggest that this may indeed be the case Ñ then it will be possible to evolve human or higher levels of intelligence by the very same methods. At least with this approach there can be milestones and benchmarks along the path to human level intelligence in a machine.

If there is any question about why one would wish to pursue these research directions, it is always possible to point to the benefits to be derived in the evolutionary, ecological, biological, ethological, and even computer science fields. But it may also turn out to be the only ÒrightÓ way to approach machine intelligence. One view of intelligence is as an evolved, adaptive response to a variable environment, that due to historical constraints and opportunism on the part of nature happens to be based upon neuronal cells. One might further recognize that intelligence is really more a near- continuum Ñ a spectrum from the simplest organism to the most complex Ñ rather than some singular event unique to humans. Then, by utilizing both the method (Natural Selection) and the tools (assemblies of neuronal cells) used in the creation of natural intelligence, PolyWorld is an attempt to take the appropriate first steps towards modeling, understanding, and reproducing the phenomenon of intelligence. For while one of the grand goals is certainly the development of a functioning human level (or greater) intelligence in the computer, it would be an only slightly less grand achievement to evolve a computational Aplysia that was fully knowable Ñ fully instrumentable, and, ultimately, fully understandable Ñ to let us know that we are on the right scientific path.

Acknowledgements

The author would like to thank Alan Kay and Ann Marion of AppleÕs Vivarium Program for their support and encouragement of this admittedly exotic research. He would also like to thank his wife, Levi Thomas, for her patience, understanding, and support throughout the project.

References

1. Ackley, D., and M. Littman (1992), ÒInteractions between Learning and EvolutionÓ In Artificial Life II, edited by C. Langton, C. Taylor, J. Farmer, and S. Rasmussen. Santa Fe Institute Studies in the Sciences of Complexity Proc. Vol. X. Addison-Wesley, Redwood City, CA, 1992.

2. Belew, R. K., J. McInerney, and N. N. Schraudolph (1992), ÒEvolving Networks: Using the Genetic Algorithm with Connectionist LearningÓ In Artificial Life II, edited by C. Langton, C. Taylor, J. Farmer, and S. Rasmussen. Santa Fe Institute Studies in the Sciences of Complexity Proc. Vol. X. Addison-Wesley, Redwood City, CA, 1992.

3. Braitenberg, V. (1984), Vehicles: Experiments in Synthetic Psychology. A Bradford Book, MIT Press, Cambridge, 1984.

4. Chalmers, D. (1991), "The Evolution of Learning: An Experiment in Genetic Connectionism" In Connectionist Models, Proceedings of the 1990 Summer School, edited by D. S. Touretzky, J. L. Elman, T. J. Sejnowski, G. E. Hinton, Morgan Kaufmann, San Mateo, CA, 1991.

5. Cliff, D. (1991), ÒThe Computational Hoverfly; a Study in Computational NeuroethologyÓ In From Animals to Animats, Proceedings of the First International Conference on Simulation of Adaptive Behavior, edited by J.-A. Meyer and S. Wilson. A Bradford Book, MIT Press, Cambridge and London, 1991.

6. Collins, R. J., and D. R. Jefferson (1992), ÒAntFarm: Towards Simulated EvolutionÓ In Artificial Life II, edited by C. Langton, C. Taylor, J. Farmer, and S. Rasmussen. Santa Fe Institute Studies in the Sciences of Complexity Proc. Vol. X. Addison-Wesley, Redwood City, CA, 1992.

7. Conrad, M., and M. Strizich (1985), ÒEVOLVE II: A Computer Model of an Evolving EcosystemÓ, Biosystems 17, 245-258, 1985.

8. Conrad, M. (1987), ÒComputer Test Beds for Evolutionary TheoryÓ Oral Presentation at Artificial Life I Conference, 1987.

9. Dawkins, R. (1976), The Selfish Gene. Oxford University Press, Oxford, 1976.

10. Dawkins, R. (1983), The Extended Phenotype: The Gene as a Unit of Selection. Oxford University Press, Oxford, 1983.

11. Dawkins, R. (1986), The Blind Watchmaker. W.W. Norton, New York, 1986.

12. de Boer, M. J. M., F. D. Fracchia, and P. Prusinkiewicz (1992), ÒAnalysis and Simulation of the Development of Cellular LayersÓ In Artificial Life II, edited by C. Langton, C. Taylor, J. Farmer, and S. Rasmussen. Santa Fe Institute Studies in the Sciences of Complexity Proc. Vol. X. Addison-Wesley, Redwood City, CA, 1992.

13. Dewdney, A. K. (1984), ÒComputer Recreations: In the Game Called Core War Hostile Programs Engage in a Battle of BitsÓ, Scientific American 250(5), 14-22, May 1984.

14. Dewdney, A. K. (1984), ÒComputer Recreations: A Core War Bestiary of Viruses, Worms, and Other Threats to Computer MemoriesÓ, Scientific American 252(3), 14-23, March 1985.

15. Dewdney, A. K. (1987), ÒA Program Called MICE Nibbles its Way to Victory at the First Core War TournamentÓ, Scientific American 256(1), 14-20, January 1987.

16. Farmer, J. D., and A. dÕA. Belin (1992), ÒArtificial Life: The Coming EvolutionÓ In Artificial Life II, edited by C. Langton, C. Taylor, J. Farmer, and S. Rasmussen. Santa Fe Institute Studies in the Sciences of Complexity Proc. Vol. X. Addison-Wesley, Redwood City, CA, 1992.

17. Harp, S., T. Samad, and A. Guha (1990), ÒTowards the Genetic Synthesis of Neural NetworksÓ In Proc. Third Intl. Conf. on Genetic Algorithms, edited by J. D. Schaffer, Morgan Kaufmann, San Mateo, CA, 1990.

18. Hebb, D. O. (1949), The Organization of Behavior, John Wiley and Sons, Inc., New York, 1949.

19. Hillis, D. (1990), ÒSimulated Evolution and the Red Queen HypothesisÓ Oral Presentation at Artificial Life II Conference, 1990.

20. Hillis, D. (1992), ÒCo-Evolving Parasites Improve Simulated Evolution as an Optimization ProcedureÓ In Artificial Life II, edited by C. Langton, C. Taylor, J. Farmer, and S. Rasmussen. Santa Fe Institute Studies in the Sciences of Complexity Proc. Vol. X. Addison- Wesley, Redwood City, CA, 1992.

21. Holland, J. (1990), ÒEcho: Explorations of Evolution in a Miniature WorldÓ Oral Presentation at Artificial Life II Conference, 1990.

22. Hugie, D. (1992), Personal Communication.

23. Jefferson, D., R. Collins, C. Cooper, M. Dyer, M. Flowers, R. Korf, C. Taylor, and A. Wang (1992), ÒEvolution as a Theme in Artificial Life: The Genesys/Tracker SystemÓ In Artificial Life II, edited by C. Langton, C. Taylor, J. Farmer, and S. Rasmussen. Santa Fe Institute Studies in the Sciences of Complexity Proc. Vol. X. Addison-Wesley, Redwood City, CA, 1992.

24. Klopf, A. H. (1986), ÒA Drive-Reinforcement Model of Single Neuron Function: An Alternative to the Hebbian Neuronal ModelÓ In Neural Networks for Computer, edited by J. S. Denker, AIP Conference Proceedings 151, American Institue of Physics, New York, 1986.

25. Klopf, A. H. (1987), ÒA Neuronal Model of Classical ConditioningÓ, AFWAL-TR-87-1139, Air Force Wright Aeronautical Laboratories, October, 1987.

26. Koza, J. R. (1992), ÒGenetic Evolution and Co-Evolution of Computer ProgramsÓ In Artificial Life II, edited by C. Langton, C. Taylor, J. Farmer, and S. Rasmussen. Santa Fe Institute Studies in the Sciences of Complexity Proc. Vol. X. Addison-Wesley, Redwood City, CA, 1992.

27. Langton, C. G. (1989), ed. Artificial Life. Santa Fe Institute Studies in the Sciences of Complexity Proc. Vol. VI. Addison-Wesley, Redwood City, CA, 1989.

28. Linsker, R. (1988), ÒTowards an Organizing Principle for a Layered Perceptual NetworkÓ In Neural Information Processing Systems, edited by D. Z. Anderson. American Institute of Physics, New York, 1988.

29. Linsker, R. (1988), ÒSelf-Organization in a Perceptual NetworkÓ, Computer 21(3), 105-117, March 1988.

30. Linsker, R. (1989), ÒAn Application of the Principle of Maximum Information Preservation to Linear SystemsÓ In Advances in Neural Information Processing Systems 1, edited by D. S. Touretzky. Morgan Kaufmann Publishers, San Mateo, CA, 1989.

31. Miller, G. F., and P. M. Todd (1991), ÒExploring Adaptive Agency I: Theory and Methods for Simulating the Evolution of LearningÓ In Connectionist Models, Proceedings of the 1990 Summer School, edited by D. S. Touretzky, J. L. Elman, T. J. Sejnowski, G. E. Hinton, Morgan Kaufmann, San Mateo, CA, 1991.

32. Miller, S. M., and L. E. Orgel (1974), The Origins of Life. Prentice- Hall, Englewood Cliffs, NJ, 1974.

33. Nolfi, S., J. L. Elman, and D. Parisi (1990), "Learning and Evolution in Neural Networks", CRL Tech. Rep. 9019, Center for Research in Language, UCSD, La Jolla, CA, 1990.

34. Packard, N. (1989), ÒIntrinsic Adaptation in a Simple Model for EvolutionÓ In Artificial Life, edited by C. Langton. Santa Fe Institute Studies in the Sciences of Complexity Proc. Vol. VI. Addison-Wesley, Redwood City, CA, 1989.

35. Parisi, D., S. Nolfi, and F. Cecconi (1991), ÒLearning, Behavior, and EvolutionÓ, Tech. Rep. PCIA-91-14, Dept. of Cognitive Processes and Artificial Intelligence, Institue of Psychology, C.N.R. - Rome, June 1991. (To appear in Proceedings of ECAL-91 Ñ First european Conference on Artificial Life, December 1991, Paris).

36. Pearson, J. (1987), ÒCompetitive/Cooperative Behavior of Neuronal Groups in Brain FunctionÓ Oral Presentation at Artificial Life I Conference, 1987. (And in Edelman, G. M. Neural Darwinism: The Theory of Neuronal Group Selection. Basic Books, New York, 1987.)

37. Rassmussen, S., C. Knudsen, R. Feldberg, and M. Hindsholm (1990), ÒThe Coreworld: Emergence and Evolution of Cooperative Structures in a Computational ChemistryÓ In Emergent Computation, edited by Stephanie Forrest, North-Holland, Amsterdam, A Special Volume of Physica D, Vol. 42 (1990) Nos. 1-3.

38. Ray, T. S. (1992), ÒAn Approach to the Synthesis of LifeÓ In Artificial Life II, edited by C. Langton, C. Taylor, J. Farmer, and S. Rasmussen. Santa Fe Institute Studies in the Sciences of Complexity Proc. Vol. X. Addison-Wesley, Redwood City, CA, 1992.

39. Renault, O., N. M. Thalmann, and D. Thalmann (1990), ÒA Vision- based Approach to Behavioural AnimationÓ, J. of Visualization and Computer Animation Vol. 1, 18-21.

40. Taylor, C. E., D. R. Jefferson, S. R. Turner, and S. R. Goldman (1989), ÒRAM: Artificial Life for the Exploration of Complex Biological SystemsÓ In Artificial Life, edited by C. Langton. Santa Fe Institute Studies in the Sciences of Complexity Proc. Vol. VI. Addison-Wesley, Redwood City, CA, 1989.

41. Todd, P. M., and G. F. Miller (1987), ÒA General Framework for the Evolution of Adaptive Simulated CreaturesÓ Oral Presentation at Artificial Life I Conference, 1987.

42. Todd, P. M., and G. F. Miller (1991), ÒExploring Adaptive Agency II: Simulating the Evolution of Associative LearningÓ In From Animals to Animats, Proceedings of the First International Conference on Simulation of Adaptive Behavior, edited by J.-A. Meyer and S. Wilson. A Bradford Book, MIT Press, Cambridge and London, 1991.

43. Travers, M. (1989), ÒAnimal Construction KitsÓ In Artificial Life, edited by C. Langton. Santa Fe Institute Studies in the Sciences of Complexity Proc. Vol. VI. Addison-Wesley, Redwood City, CA, 1989.

44. Walter, W. G. The Living Brain. W. W. Norton, New York, 1963.

45. Walter, W. G. (1950), ÒAn Imitation of LifeÓ, Scientific American, 182(5), 42-45, May 1950.

46. Walter, W. G. (1951), ÒA Machine that LearnsÓ, Scientific American, 185(2), 60-63, August 1951.

47. Wharton, J., and B. Koball (1987), ÒA Test Vehicle for Braitenberg Control AlgorithmsÓ Oral Presentation at Artificial Life I Conference, 1987.