The evolution of meaning – from pragmatic couplings to semantic representations.
When living creatures perceive something, they’re concerned with two questions: What is it? and: What should I do about it? You might think that the machinery for answering those questions evolved in that order – like you’d have to know what something is before you can know what to do about it – but it seems likely to have been the opposite. The actions of the simplest creatures when faced with various stimuli in the world are mostly coordinated by pragmatic couplings – signals that are prescriptive rather than descriptive. But these mechanisms laid the foundation for the evolution of decoupled internal representations with true semantic content.
For living organisms to go on persisting – which, let’s face it, is their whole schtick – they have to take in energy and raw materials (food, oxygen) and use them to keep their internal economy humming. Many organisms manage this process – known as homeostasis – by staying put and letting resources come to them. The problem with this strategy is that sometimes environmental conditions change and resources dry up – this is especially true if the organism’s own activity (along with its friends) uses up resources locally. A good strategy under such circumstances is to move, and many creatures evolved various ways of doing that – swimming, sliding, oozing, crawling, rowing, wriggling, eventually walking, running, and even flying.
But where to go? And how to know?
Some creatures – like a lot of marine plankton – simply float about on ocean currents and hope for the best. Others sense a depletion of food or a build-up of toxic waste products in their current environment and simply set off in a random direction – anywhere else being better than here. But most have evolved some kinds of specific sensors to detect relevant substances or objects in the environment in order to control or inform the direction of movement.
Some of the best-studied examples of these systems are in the simplest organisms – such as the chemotaxis responses in bacteria like E. coli. These rod-shaped bacteria move about by rotating a long thread-like filament called a flagellum that extends from one end and works a bit like an outboard motor. It has an unusual mechanism of action, though – when it rotates in one direction, it recruits a bunch of other filaments on the surface and forms one big propeller, pushing the bacterium forwards. But when it rotates in the other direction, all those other filaments interfere with it and the whole bacterium just revolves on the spot.
Most of the time, E. coli will swim in a straight line for a bit, then tumble around, and then head off in a new, apparently random direction. However, when they detect some chemical substance that is a food source – like glucose, for example – they will spend more time swimming in a straight line, as long as the signal from the detection of the food source is increasing. In this way, they travel up the concentration gradient and arrive at the spot with the maximal amount of resources.
The molecular factors that mediate this behaviour include receptor proteins that protrude like antennae from the surface of the bacterium and can bind to the food substances, and internal proteins to process and transmit the signal, ultimately controlling the direction of rotation of the flagellum. This system is often described in purely mechanistic terms, as if the bacterium were a passive, stimulus-response machine.
This is deeply mistaken – the bacterium is an endogenously active agent, constantly monitoring its environment for information that it accommodates to. In isolation, the system I just described looks very linear and deterministic – as if its activation will push the bacterium one way or another. In reality, it works in a much more integrative and holistic manner. E. coli have receptors for many different kinds of chemicals and other stimuli, including potentially harmful ones. These must all be integrated to “compute” the most adaptive direction to travel. And the signals from any potential food source must be compared over adjacent time-steps for the bacterium to be able to follow a gradient. In addition, all of this machinery can be modulated by conditions like temperature and osmolarity and the overall density of other cells.
So, there is a lot of integrative processing going on even in these apparently simple actions of very simple creatures – so much that it is sometimes referred to as “basal cognition”. However, there are some crucial differences from the kind of activities typically thought of as cognitive in animals or humans.
The main difference with fancier types of cognition is in what those signals mean to the organism. And here we get into some tricky philosophical waters. We’re used to talking about information in scientific terms – it can be localised and even quantified. Meaning is a slippier concept. It’s obviously much more qualitative and subjective – more constructed by the interpreter of a signal than residing somehow in the signal itself.
We can say that the activation of a protein receptor molecule conveys information to the interior of the cell – information about something out in the world. But it is also information for something. The reason bacteria have these receptors is that it is adaptive for them to be able to detect these substances out in their environment and do something about it. In fact, these simple systems are configured in such a way that the organism just does something without separately apprehending what the signal is about.
The meaning to the organism is not “there’s glucose here” or even “there’s food here” and it’s not “there’s a dangerous chemical here” or even “there’s a threat here”. The meaning of these signals to the organism is simply “approach!” or “avoid!”. They are prescriptive, not descriptive. Or, if you prefer, they are imperative, not indicative. They’re not pointing at something or reporting what’s out there – they are simply demanding action.
So, in these simple cases, the organism doesn’t infer what’s out in the world and then decide what to do. Really, there’s no inference going on at all – just an adaptive response. The meaning of these signals is not (yet) in their semantic content or aboutness – for living organisms, meaning started with salience. The information is meaningful for some entity, relative to some goal or purpose.
For living creatures, that purpose is staying alive. This is the master function that anchors everything else. It emerges straightforwardly – inevitably – from the action of natural selection. Organisms that are better at persisting, persist better. Their structures come to be configured in ways that are best fitted to surviving and reproducing in their environments. This isn’t just a passive kind of persistence, either – organisms have to do thermodynamic work to stay alive, and evolution does design work to favour ones that do it best.
This adaptation includes the configuration of the systems that integrate sensory signals and control behaviour. Creatures that tend to approach food sources and tend to avoid danger will do better than ones that don’t. These stimuli therefore have value, relative to the goal of persistence. And they come to have meaning – “approach!” or “avoid!” – wired right into the biochemical configuration, through the cumulative verdict of natural selection on the types of behaviours that organisms take in response to them.
The meaning is thus not just in the “content” of the signals. The currently active signals conveyed by receptors for these various stimuli are interpreted in the context of stored control policies that are wired into the system.
That’s all well and good – those kinds of facilities enable simple creatures to navigate their world adaptively. But they are limited in scope. This is because all that processing is happening at one level, within a single, shared space over a common timeframe – everything, everywhere, all at once. You can do a lot by merging a bunch of different signals like that but you quickly run out of degrees of freedom. There is just a limit to how many cogs you can compute with in a single, fully interconnected system before they get jammed up. The solution to this problem is to introduce some intervening layers and decouple the systems of perception and action. This is what nervous systems are good for.
Big beasts with brains
When multicellular life emerged, it faced a new problem. Animals now had bodies with lots of different bits – bits that all had to be coordinated for the animal to be able to move in an effective fashion. Muscles evolved to move the bits and neurons evolved to help coordinate them – with each other and also in response to sensory information. At first, these functions may have been performed by the same cells – ones that sat in the skin, with an outer sensory part and an inner “muscular” part, capable of contracting. But at some stage, these jobs were split into sensory cells and motor cells, and then a new type of cell came along, which sat between them – neurons.
The advantage of having this intervening layer of neurons is that they could communicate across a whole field of sensors and motor actuators to coordinate them. And they could do it fast. The biochemical signaling and processing that happens in single cells is powerful and efficient, but it’s slow and not suited to communication over long distances. Neurons use electrical signals instead, which are extremely rapid, and they evolved complex shapes and long projections that could connect elements in different parts of the organism. Organisms thus evolved new, powerful systems for behavioural control and a new language for internally representing and processing information.
As nervous systems got more complex, they started to add more and more internal layers, between the sensory and motor systems. It’s important to note that there are still some systems where sensory signals are rapidly coupled to motor outputs, with a minimum of processing. For example, many insects and fish (and even some mammals) have a hard-wired escape response to certain visual stimuli, such as a big shadow passing overhead or looming towards the animal. It’s pretty obvious how such a coupled sensorimotor system would remain adaptive – it gives speed and reliability to this crucial survival response. But the real benefits of complex nervous systems come with the decoupling of perception from obligate action, which, by contrast, gives greater flexibility and integrative control.
Adding layers of neurons also increases the amount of processing that can be done on perceptual information, enabling organisms to extract higher-order information about what is out in the world. The earliest-evolving senses were smell and touch – the detection of chemical or mechanical stimuli directly in contact with the organism. Creatures that rely solely on these kinds of senses cognitively and behaviourally inhabit the here and now – they have no access to things that are far away from them, spatially or temporally, and, as a consequence, have not developed the cognitive resources to think about them or act with respect to them.
The evolution of vision and hearing changed that. These are distance senses – they’re designed to detect disturbances in the medium around an organism, whether that is the electromagnetic spectrum or the vibrations of air or water. This allows organisms to indirectly infer the presence of objects – most importantly other organisms – that are out in the world and that may be the causes of those disturbances. This is not a simple task, however – organisms have to solve the “inverse problem” and figure out the most likely causes of patterns of ambiguous stimuli.
This is where those intervening levels of neurons earn their keep. By integrating inputs from the level below, each new level extracts higher and higher-order information. In the visual system, for example, the first levels in the retina are just responding to photons of light. But the next levels are comparing across inputs, performing contrast enhancement, and feature extraction, making inferences about edges and orientations and movements. And as that information is processed in the brain itself, further inferences are drawn about objects, where they are, what they are, and how they’re moving, relative to each other and the organism itself.
All of that information is made available to the rest of the brain to inform action as appropriate, given whatever else may be happening in the world, what the current state of the organism is, what its goals are, and so on. This kind of decoupling from action thus provides a much more flexible control of behaviour, guided by richer and deeper cognition, operating over longer timescales. It pays to think about the future when you can literally see things coming from a mile away.
The patterns of neural activity now constitute internalised representations that are “detached” from obligate action. Their job is no longer imperative, but indicative, or declarative. Whereas in simple organisms, the meaning is pragmatic, in more complex organisms, it becomes truly semantic. We can think of these systems as being configured as follows:
- Simple, pragmatic couplings: If A à do X.
- Complex, semantic representations: If A à then “A” – as in, tell everybody that: “A”.
That information (that “A” is the case) can then be used to inform all kinds of further cognitive operations, inferences, and possible actions. It’s a “see something, say something” sort of scenario. The contrast between these scenarios is like the difference between a thermostat and a thermometer. A thermostat detects and in the same process acts on information about the temperature. A thermometer merely reports it (or represents it, to anyone that is bothered to look at it).
Our visual system does the same thing – it reports its inferences about what is out in the world to the rest of the brain (and to the organism as a whole). But this creates a new problem. Now that these signals are internalised, how do downstream “users” (either other neurons or brain regions or the organism itself) know what the signal or pattern is referring to? How is the semantic content anchored?
No naked representations
The meaning can’t be just given, by the pattern itself, in isolation – that’s just some neurons firing. As experimenters, we may know that a given pattern correlates with something out in the world (which we can see independently). But how do other parts of the brain, or the whole organism know that? They can’t see the thing – they only have access to the pattern of neurons firing. That pattern can’t entail its own meaning. That would be like looking up the word “car” in the dictionary and finding the definition: “car”. It can’t define itself. The meaning of the word “car” is instead given by other associated words that describe its properties and relations: “a four-wheeled road vehicle that is powered by an engine and is able to carry a small number of people”.
A similar process happens in the brain. The accumulation of such associations starts in infancy, as babies explore their world, especially by cross-calibrating between their senses. They learn that things that look like this, feel like that and taste like that and are hard and about this heavy and I can pick them up and if I drop them they make this kind of a sound. All of us gradually build up a web of knowledge of these kinds of properties and affordances of objects – latent schemas that are activated or at least accessible when we detect an example of that type of object or even when we think of it. In this way, percepts are grounded on the basis of stored concepts. And those concepts can get progressively more abstract, encompassing hierarchical categories, causal relations, and narrative sequences of events – all the information that an organism needs, in order to decide what to do in a given situation.
The semantic content of a given representation is thus embodied in a web of associations – a set of linkages and pointers to other characteristics or properties, themselves represented in latent structures across the brain. The representation is thus discrete, but its meaning is distributed. We saw with pragmatic couplings that the meaning of currently active signals is given by the context of stored control policies. With semantic representations, the meaning of any current pattern of neural activity is given by the context of stored knowledge. All of this is calibrated through experience, rigorously selected for salience, and ultimately still used to inform action, just over many more possibilities and much longer timescales.
The organism still wants to know what it should do. But now it uses decoupled internal representations (and stored knowledge) of what is out there to inform those choices.