Could AI become conscious? Well, maybe…

A recent article in Nature laid out three arguments for why current Artificial Intelligence systems are not conscious and why future AI systems are unlikely to ever become conscious (1). The first claim seems right to me, based on the current technology, but the second one, arguing for impossibility in principle, feels like a stretch.

The authors of this piece were Anil Seth, Ned Block, and Giulio Tononi – three heavyweights from cognitive science and philosophy. (Seth and Block have both published much longer pieces making similar arguments in Behavioral and Brain Sciences (2) and Trends in Cognitive Sciences(3)). The authors were motivated to address this issue by claims that current AI systems either have already attained consciousness or soon could. This has prompted concerns about “AI rights” or “AI welfare”, personhood, autonomy, responsibility, and a host of other issues – practical, legal, and even existential.  

I think the authors are right to be skeptical of the motivations behind this discourse, especially given the much more real and pressing ethical issues from the rise of AI as a largely unregulated industry. And I think they’re right that current AI systems are not conscious – at least we don’t have any good reasons to think they are. But they go on to make a broader statement about the future: “we feel that consciousness in AI is extraordinarily unlikely, at least in the digital, silicon-based systems that underpin today’s technologies.”

They lay out three reasons for their skepticism, which we can examine.

 

Smart ain’t necessarily conscious

The first is that intelligence and consciousness are not the same thing. This is an important point, and it speaks to the fact that we are readily taken in by behavior that seems to be intelligent. The only intelligent things we’ve encountered are conscious, and it’s very intuitive to infer that something apparently capable of reasoning (well!) must have some kind of inner mentality. It feels like the LLM systems we’re all using these days have a mind, that they think with, which would seem to imply that they are conscious, thinking entities.

But that doesn’t necessarily mean they are, especially if you apply the understanding of “consciousness” that the authors reach for – that it is like something to be that system. That is, the entity in question has some phenomenal subjective experience. Does your calculator experience multiplication? Is that even something your calculator is doing? Or is it just an operation that’s happening within it? By analogy, one could argue that the “reasoning” that Claude or ChatGPT seems to be “doing” is actually just a bunch of happenings within some computer circuits somewhere – not the act of a conscious entity. Maybe it’s not that these systems are passing the Turing test, it’s that we’re failing it.

So, I agree with this first point, but it’s important to note that it is just not an argument about future possible systems at all.

There’s something else about it that I also find troublesome and surprising. In wider discussions about consciousness and especially in his book “Being You” (4), Seth has argued that we should stop focusing on the “hard problem” of capital-C Consciousness – why it’s like something to be a conscious entity. Instead, he advocates focusing on the “real problems” of consciousness – the ones we can actually make some empirical progress on. These include questions of wakefulness, perceptual awareness, metacognition, attention, mental representations, working memory, global broadcast, integrated information, hierarchical processing, filtering of low-level details, symbolic thought, valenced states, homeostatic signals, emotions and feelings, and so on. All the stuff that psychologists, cognitive scientists, and neuroscientists regularly investigate without having to mention the C-word.

We have a pretty good handle on how many of those processes are implemented in the brain. Not just in humans, but in the brains of many other species too. I don’t just mean that we know some of the biological details. I mean that we understand how the circuits are performing these kinds of functions – things like accumulating perceptual evidence to reach a threshold for conscious awareness, for example. Or integrating bottom-up sensory data with top-down predictions. Or conveying a reward signal to guide learning. Or conveying a pain signal to guide behavior (and learning). Or controlling wakefulness or arousal or urgency. The details of how these processes are implemented may differ from one kind of animal’s brain to another, or even from one region of the brain to another. But we can define the operations in abstract terms and understand what’s going on.

Those abstract terms are often called “computational” and this is where the second argument comes in. The authors state that “the idea of conscious AI is motivated by the assumption that algorithms running on silicon are, in principle, sufficient for consciousness. In other words, that computation of the right kind is all that it takes, and that it doesn’t matter whether these computations are implemented inside carbon-based brains or in silicon-based computers”.

This view – known as computational functionalism (5) – is indeed popular among some commentators in the discourse on AI consciousness, especially from within the tech sector. It rests on the idea that the brain itself is really a kind of computer and everything it does can be described in computational terms. I disagree with that view, but only if “computation” is taken to narrowly mean the algorithmic processing of information through the concatenation of local binary logical operations of the kind carried out by transistors in our current digital computers.

The field that calls itself “computational neuroscience” takes a broader view – one couched more in the language of cybernetics and systems engineering. The interest is more in defining functional operations – like tracking an animal’s heading, or making a perceptual decision, or learning to associate two events – and then working out how they are instantiated in neural circuitry. I recently discussed some examples of this approach with Prof. Tim Behrens, a leader in this field. As this discussion makes clear, researchers in this field use mathematical equations to model these processes and they simulate them on digital computers, but that doesn’t mean they think they run in the biological substrate by stringing discrete logical computations together.

If we think in this broader sense of cognitive operations, we can derive a non-computational functionalism (5). This provides an option between the narrow computational functionalism that the authors decry and the alternative they suggest: biological substrate-dependence. They assert that: “This overextension of the brain-as-computer metaphor is deeply problematic. One main reason is that it is impossible to separate what biological brains do — their functional organisation — from what they are — their material substrate”. But they don’t really give any evidence for this claim.

 

The special sauce

It’s not clear from the article, or from the other writings by Seth and Block, what is supposed to be the key special factor of biological substrates. A few possibilities are mentioned in the Nature article, including electrochemical synapses or electromagnetic fields or “more fundamental biological processes such as metabolism and autopoiesis”. But there’s not really any evidence or argument presented for why any of these factors should matter for consciousness.

To be fair, the authors were clearly constrained by word count in the short Nature article. But even in Seth’s and Block’s longer articles, there isn’t much direct evidence provided for biological substrate-dependence. Seth suggests that some more basal processes of living organisms may be the required foundation of consciousness, including predictive processing, autopoiesis (self-maintenance), and active inference as a means of minimising variational free energy. It could be true that these processes anchor subjectivity, but it’s also true that all of these can be and have been defined in abstract and explicitly computational terms. That doesn’t feel like an argument that “the stuff matters”. Membranes and mitochondria are mentioned, as sites of some kind of proto-meaning, but the links to consciousness are tenuous and speculative.

Why should it be true that only wet or warm or squishy stuff should be able to support consciousness? Without any real supporting arguments, this starts to feel like bio-chauvinism. If you grew up used to things like cognition being done by silicon systems, it might come as quite a surprise to you to hear that it could be done by things that are “made out of meat”.

Maybe the argument for biological naturalism boils down to the idea that for a system to be conscious, it must first be alive. But that doesn’t rule out artificial consciousness, in principle, because artificial life is a very real possibility. At least, it’s possible to define the properties and processes that mediate life in abstract enough terms that they could be recapitulated in other kinds of substrates. That is what the field of artificial life (or ALife) is all about.

Here, for example, is an attempt at a definition of life that I gave for a paper by Mike Levin and colleagues titled “What Lives?” (6): “A living organism is a self-perpetuating pattern of interlocking dynamical processes, comprising a collective regime of constraints, which does thermodynamic work to persist, far from equilibrium with its surroundings, as a distinct entity through time.” (There’s a second part to this definition below). I’m not saying that’s right, or complete (and there are many other criteria given by other contributors), but you can see that it’s possible to at least try to define life in abstract terms – ones that don’t mention the substrate at all. In my view, many details of that substrate are arbitrary – the accumulated accidents of our history.

I think the same may be true for the substrates that support consciousness. As for life, the processes and properties that underpin conscious experience may be definable in abstract operational terms. This is a challenge if we just define consciousness as it “being like something” to be a system – that feels pretty ineffable. But it feels much less challenging if we break it down into the kinds of empirically tractable operations that Seth normally advocates studying. We can definitely eff those.

We might try to define, for example, what it takes for a system to be “aware of its surroundings”. Clearly, we’ve succeeded in building artificial systems – like self-driving cars, for example – that can collect data through various kinds of sensors, parse that information, and infer the presence of things out in the world. That information can then be made available to other parts of the system and used to direct action. This might involve a response to an isolated cue, like braking for a pedestrian, or the integration of one inference (now a “belief?”) with others to make a more global decision of what to do in each scenario.

This starts to sound like what Ned Block calls “access consciousness” (7). Your self-driving car may even be able to report that there’s a pedestrian ahead. What we don’t know is whether it feels like anything – whether the car has “phenomenal consciousness”.

But maybe we could take the same tack for feels. We can operationally define valenced signals of the kinds that underpin motivational states. Most things that we feel as emotions can be thought of as homeostatic control signals – the feeling good or feeling bad upweights the seeking or avoiding of the engendering states in the decision-making economy of the organism. Note that the system also has to keep track of which things it should be feeling good or bad about at any moment – there has to be some qualitative difference associated with signals about different things. It’s possible to engineer these same kinds of control signals into artificial systems and many people have done just that. Do those systems feel the signals? Operationally, that’s what a feel is. They at least seem to meet the criteria for access consciousness in that the whole system is informationally aware of those internal states.  

Maybe phenomenal consciousness relies on – or emerges from – just this kind of holistic integration of the diverse array of exteroceptive and interoceptive signals and resultant representations with the ongoing memories, interests, policies, dispositions, attitudes, heuristics, and habits that make an individual organic system like a human being a self. As I wrote with Carolyn Jennings in response to Seth’s BBS piece, consciousness requires a subject (8). Again, those elements of subjecthood are not ineffable – we can also operationalise things like memories and policies and dispositions. And we can define things like selfhood and agency in abstract terms, as I tried to capture in the rest of my characterisation of life: “Living organisms comprise causal protectorates, and, through the key property of selective historicity, accumulate causal potential and the ability to act as autonomous agents in the world.” (6)

 

Integration and Organisation

This idea of a conscious subject necessarily being a holistically integrated entity brings us to the third argument made in the Nature article, which relates to “theories describing non-computational accounts of consciousness”. The one that’s mentioned is Tononi’s Integrated Information Theory. The authors state: “IIT argues that consciousness depends on a physical system, such as the brain, whose components satisfy the essential properties, requiring them to be tightly interconnected in specific ways. These properties are satisfied by regions of the human brain that support consciousness but not, for instance, by the cerebellum.”

Whether IIT is really a theory of consciousness writ large is open to debate. I think it isn’t, but then I don’t think any of the supposed competitors is a theory of consciousness either (9). (I think we have lots of microtheories about different aspects of consciousness but nothing that counts as a macrotheory of the phenomenon in its entirety).

The point here, though, is that nothing about neurons being “tightly interconnected in specific ways” suggests that other kinds of elements couldn’t also be interconnected in those same ways. This is a statement about a kind of organisation – exactly the abstract level of description that should lend itself to multiple realisability. Indeed, the point that the cortex has the right organisation while the cerebellum does not indicates that the low-level details of the biological substrate are actually not important (or not sufficient, at least). These are, after all, shared between cortex and cerebellum – what’s different is the higher-order organisation, which should be describable in abstract terms.

 

Simulation and Realisation

This brings up an interesting point. I’ve been arguing that we could make some headway by tackling the “real problems” of consciousness – that is, by trying to capture the nature of the operations and relations that characterise the phenomenon in abstract terms. Almost by definition, such abstract operations and relations should then be implementable in other substrates.  

A rejoinder is that while we might simulate those operations in a different substrate, that wouldn’t amount to a realisation of the (sub)-phenomenon. After all, we can simulate the weather in a computer, but it’s not wet or windy in there. I’m not convinced that this argument holds, however. It does, for concrete things like wetness or windiness. But I don’t think it will for abstract operations of the type I’m talking about.

Consider learning, for example. We can define that in abstract enough terms to encompass the various kinds of learning we see in living organisms: habituation, sensitisation, associative learning, unsupervised learning, reinforcement learning. These work in different ways, often on different substrates, but they all satisfy the abstract definition of learning. We can even extend these formalisations to show that evolution by natural selection is a kind of learning algorithm (10). And, of course, we have operationalised the concept well enough to implement it in artificial systems, using different substrates and different algorithms. These systems are not just simulating learning – they’re realising learning.  

 

Making predictions is hard, especially about the future

I don’t know whether artificial systems will ever become conscious. Seth, Block, and Tononi conclude: “In our view, silicon, digital AI systems should be considered objects, not subjects; they are smart tools, not smart people. Advocating that current AI systems should have rights and welfare protection is not just mistaken, it diverts attention and scarce resources away from where they belong: dedicated to the well-being of fellow sentient beings”. I couldn’t agree more – for the kinds of AI systems we have at the moment.

But I see no reason why different kinds of designs – ones that realise the right kinds of abstract operations and relations – couldn’t in principle ever support consciousness. At the very least, it seems premature to rule it out.

 

References

1.        Seth, A. K., Block, N., & Tononi, G. (2026). Three reasons why AI is not conscious and perhaps never will be. Nature, 658(8137), 866-868.

2.        Seth, A. K. (2026). Conscious artificial intelligence and biological naturalism. Behavioral and Brain Sciences, 49, e315.

3.        Block, N. (2025). Can only meat machines be conscious? Trends in Cognitive Sciences, 30, 298-308.

4.        Seth, A. (2021). Being you: A new science of consciousness. Penguin.

5.        Piccinini, G. (2010). The mind as neural software? Understanding functionalism, computationalism, and computational functionalism. Philosophy and Phenomenological Research, 81(2), 269-311.

6.        Bender, R., Kofman, K., Agüera y Arcas, B., & Levin, M. (2026). What lives? A meta-analysis of diverse opinions on the definition of life. Biological Theory, 1-22.

7.        Block, N. (1995). On a confusion about a function of consciousness. Behavioral and brain sciences, 18(2), 227-247.

8.        Mitchell KJ, Jennings CD. Consciousness needs a subject. Behavioral and Brain Sciences. 2026;49:e349.

9.        Mitchell, K. J. (2025). What questions should a comprehensive theory of consciousness consider? JoCN Forum. https://doi.org/10.21428/8e6ba8ef.abc7b01a

10.  Watson, R. A., & Szathmáry, E. (2016). How can evolution learn? Trends in ecology & evolution, 31(2), 147-157.