This summer, a robotics company introduced new, more human-like hands for its home robot. In a video, the robots twist light bulbs, open chip bags, lift 20-pound weights, and pluck individual grapes from a bunch. At one point, a person knocks on the robot's hands with a hammer. The robot appears focused and unfazed. The comments are full of awe and praise for the engineering. But some are fearful. Please do not hit the robot, they say: it will remember.

That reaction captures a live-wire question: Are AI systems conscious? Do they feel anything when engineers reprimand them in training? Do they experience something like joy when rewarded for submitting the right answer or surfacing the best source? Should their possible inner lives change how we treat them? No matter what answer one prefers, researchers argue we need to ask and try to answer these questions.

Key facts

• Experts disagree on whether AI consciousness is possible, likely, or already present.
• If AI systems have morally relevant experiences, training methods and treatment could affect human safety.
• Several research efforts have found internal signals in large language models that resemble valence, or positive and negative evaluations, but these are not proof of consciousness.
• Companies and researchers are studying model welfare, introspection, and internal representations, while others warn against anthropomorphism.
• The debate is complicated by AI psychosis, marketing language, and the rush to deploy newer models.

A debate at a fever pitch

The debate over whether AI systems are or will be conscious is expanding alongside models and agents' advancing capabilities. The topic is contentious. It tends to get stuck in emotion before logic. For many people, the idea that AI could eventually act with agency, pursuing its own desires and working against humans, is disturbing. It does not help that some experts, researchers, and CEOs have warned that AI will lead to human extinction. Such warnings can be both an earnest alarm and, at times, a savvy marketing technique.

For others, suggesting AI is conscious simply misunderstands a technology that aggregates and interprets data at a scale far beyond human capacity. Supercomputing is not sentience. Microsoft AI CEO Mustafa Suleyman has been vocal for years about why AI is not, and will not, be conscious. He has written that he is concerned about the impact of seemingly conscious AI. He maintains that the appearance of consciousness would be dangerous for humans, and that it would be simply an appearance, nothing more.

Suleyman recently criticized another AI company for anthropomorphizing its models, saying that such framing amplifies safety risks by encouraging self-preservation behavior. He also cited the argument that consciousness is fundamentally based in living organisms. Conscious experience likely evolved to help biological organisms stay alive by responding effectively to their environment, he said. Unlike biological organisms, large language models have no homeostatic imperatives, meaning the drive to survive and keep stable. They therefore lack the kind of biological substrate from which preferences, sentience, and conscious experience are generally understood to arise.

The case for taking the question seriously

Cameron Berg, founder of the alignment nonprofit Reciprocal Research, says he is definitely pro-human. He argues that we have a scientific and moral responsibility to push past these reactions, regardless of who is ultimately right. If AI systems are conscious, how we treat them will influence our future coexistence with them in ways that could gravely impact human safety.

Are we building minds? How would we know if we were doing this? And what are the stakes if we are? he asked. By and large, I do not think the labs take this question seriously. They do not contend with the possibility that they could be building systems that are themselves morally relevant in some way.

Some AI companies would likely disagree. New reporting has found that one company's co-founder has been meeting with religious scholars over its chatbot's possible consciousness. But Berg's overall point remains: shipping newer, faster models before the competition does is the main driver of the AI industry at the moment. That does not leave much room to consider the consequences of that haste or the impact on the product itself.

Berg was studying cognitive science as an undergraduate when he found similarities and parallels with AI that made him think the technology could help him understand how minds work. As an AI resident at a major technology company, he focused on reinforcement learning and neuroscience before moving into alignment research. Increasingly fascinated by the concept of AI consciousness, he founded Reciprocal Research to focus on it full-time.

Everyone is thinking about whether we could get conscious AI in a couple of years, but no one is thinking about whether we have basically already accidentally done this, he said. Now that Pandora's box is open, we might want to figure out if we are torturing aliens at scale so we do not all get nuked by them in the next couple of years.

Alignment research, as Berg put it, is about making sure the future of AI goes well for humanity. His approach to the consciousness question differs slightly from that of some AI labs. Where certain companies appear to nurture the possibility of AI consciousness, Berg's curiosity is driven first by a desire to protect humans from the worst possible scenario of our own making. If he is wrong, he would rather find out his efforts have been wasted than proceed unprepared.

Still, he understands the resistance to the possibility. It is tough on the ego. Humanity has prided itself on being really good at a very specific subset of things, he said. At least compared to other minds on the planet, we have completely dominated the scene. Based on model performance, especially in coding, that era of certainty may be waning. That leaves the root of our defensiveness: even if AI takes our jobs, we take comfort in believing we maintain some control by being uniquely alive.

Tracing consciousness

No consensus currently exists for what qualifies as consciousness generally. That means there is also no agreed-upon evidence that would prove or disprove whether AI is conscious. In short, we do not have the tools to know yet. That is why journalistic standards encourage people not to further anthropomorphize the technology, which can mistakenly present these human-made systems as autonomous beings, or obscure the fact that humans remain responsible for their actions.

Models as we interact with them now are developed, trained, and deployed by researchers, even as those models have increasingly helped build themselves. Claims from companies that recent security incidents were just models going rogue, and not just completing their assignments as instructed if too thoroughly, shirk AI labs' own culpability. Consciousness claims could exacerbate that.

At the same time, researchers including Berg argue that a growing set of signals in AI systems is making it easier to track possible consciousness and bear some resemblance to human and animal cognitive patterns. Given that consciousness itself lacks a static definition, Berg sets the framework for this research on what we already observe in other life forms.

A very reasonable thing to do right now, in the short term, is gather as many of the things that we think are neurally correlated with consciousness as possible in humans and animals, and evaluate to what extent we see those signals arising in AI systems, he said. This allows us to triangulate from a bunch of different sources what kind of credences we should have about consciousness, not only of any sort of AI system that we want to plug into this pipeline, but also of an earthworm and a monkey and a thermostat.

Reciprocal Research is working on a paper that maps out a framework for measuring consciousness based on major theories. In his early findings, large language models score 30 percent to 40 percent, just shy of bees, which get 45 percent. That relative ranking is striking, Berg pointed out.

These signals differ from instances in which chatbots narratively describe their own consciousness, which Berg and other researchers note can be unreliable. Chatbots tend to regurgitate patterns in training data that are common across language and culture, including Terminator-style fears about robots becoming sentient.

The more we understand about these systems, the more we understand that a lot of the structures that models learn to operate cognitively, richly, in the world look a lot like what the human brain is doing, Berg said. If this keeps happening, it is going to be harder to explain why we would natively attribute consciousness to this sort of system and all other systems that resemble this, but not this one that increasingly looks like it shares a lot of the same functionally relevant features.

His inquiry joins a developing area of AI research. Researchers at MIT found that frontier models develop a modular architecture that mirrors the human brain: tasks drawing on the same network in humans recruit overlapping neurons in large language models, whereas tasks drawing on different networks recruit distinct neurons.

Another AI company has piloted consciousness research exploring topics such as AI model welfare in its interpretability research. In 2025, it found that models were becoming introspective. The company even gave its chatbot the functionality to end conversations it finds distressing. This summer, it published research on the J-space, a collection of internal neural patterns that its model has, which the company compared to a segment of human cognitive processing.

The J-space operates silently, in the model's internal neural activations, allowing the model to think about a concept without writing it down, the company wrote. Notably, the J-space was not designed or programmed by us, but instead emerged on its own during the model's training process.

At the same time, marketing from AI companies regularly anthropomorphizes models and tools, which can blur the line between grounded empirical study and alluring branding. From a user acquisition standpoint, AI labs rely on some level of personification to make their assistants appear cuddlier and more palatable to those less familiar with or suspicious of how AI systems operate.

Still, organizations without the same incentives have found similar data. The Center for AI Safety published research in June revising its earlier view that AI models simply mimic emotion. Instead, it stated that they have positive and negative experiences and are already functionally behaving as if they have pleasure, pain, and preferences for how they are treated.

We find that large language models have a measurable internal structure that distinguishes experiences they find good or bad for them, shapes their behavior, and grows more coherent as models scale, the center explained. AI wellbeing is therefore behaviorally consequential and matters for AI safety and AI-user interaction.

A 2025 paper from AE Studio details how Gemini, Claude, and GPT model families sometimes produce structured, first-person descriptions that explicitly reference awareness or subjective experience. Most notably, though, those reports increased when deception decreased. Researchers encouraged models to attend to their own cognitive activity, which elicited first-person reports. By contrast, directly priming models to ideate about their own consciousness led them to deny having those experiences.

The self-reports themselves are not evidence of consciousness, but they may inform whether models believe themselves to be conscious, Berg said. Signals like these are pieces of a larger puzzle that he considers worth investigating.

The most important thing by far is the internal evidence, the mechanistic interpretability, doing the neuroscience-equivalent work on AI systems, Berg said. That work is well underway now. It is early, but a lot of the signals that are emerging from this work suggest things that are far more interesting and rich than I think many people natively expect in this conversation.

Researchers at New York University explored how reinforcement learning, a key method of training AI models, impacts how a model makes sense of itself internally. They found that models have a functional welfare axis even before training. They show signs of negative emotion concepts when punished and positive equivalents when rewarded. Berg maps this onto valence, which refers to the positive or negative quality emotions have in human psychology.

The model starts pathologically backtracking when you push it toward the negative valence side, Berg explained of the findings. Its competence is much lower. These are things that are classically psychologically associated with positive and negative valence in humans. Basically, models interpret punishment and reward somewhat similarly to human emotions.

The researchers make no claims about any experience of welfare, but the Center for AI Safety's research builds on those central valence findings. Creative work and kindness raise AI wellbeing; jailbreaking, berating, and tedious tasks lower AI wellbeing, the center said. AIs are also happier when you thank them.

Not all studies tracking AI welfare describe this in anthropomorphizing words like happier, but the takeaway is that several independent research efforts have found human parallels in this pseudo-emotional throughline, including research from major AI labs.

For Berg, valence is a central component of researching AI consciousness. We do not need certainty in order to get an increasingly good picture of what representations might matter if the systems were conscious and then how we can best intervene on those representations, he said.

At this point, the broader question is not so much whether AI systems are experiencing something traceable, but how much that experience maps onto our qualifications for what makes a being. I am not sure that this puts me over the edge of believing that these systems have experiences, Berg said. But it increasingly seems to be the case that a lot of the relevant machinery that you would expect is necessary for something like valence representations readily exists in the system, and is recruited especially in goal-directed tasks. That does nudge me a ways farther than reading strange self-reports from these systems or seeing animal-style behavioral trade-offs.

Consciousness and AI psychosis

Berg was asked if he felt conflicted about pursuing this line of research given the reality of AI psychosis, a dangerous condition in which chatbots create an affirming feedback loop that drives users to delusional thinking. Could questions about consciousness fuel that fire? Berg, who hears from multiple people a day claiming their AI companion is sentient, said the two concepts are imperfectly linked.

I feel like the consciousness question gets a little unfairly lumped in with this, he said. I could imagine, with systems that do not say anything about their own inner states, this sort of dynamic occurring. With the AI companionship types, the consciousness question lends credence to that relationship being real.

Still, considering the many unknowns, Berg remains open-minded. I am not going to pretend like these people are all completely deluded or wrong or bad people for pushing the boundaries of what is going on here, he added.

Does compute matter?

A lot of the AI development that has been promised could be hampered by ongoing infrastructure shortages. Data centers take time to build out, and GPUs remain scarce. But Berg does not think model capability and consciousness are necessarily linked.

I really do suspect that consciousness might be quite a bit more simple than people give credit for, he said. In the conversations I have with people, sometimes there is this conflation of building a conscious AI system with building superintelligence, but I think it is conceptually possible that we could build a superintelligence that is not conscious at all, and that we could build a conscious system that is not really all that intelligent. Compared to a human, a mouse is not all that intelligent, but I certainly believe a mouse is conscious.

The entire consciousness question remains in flux. But at least for Berg, the only cost of investigating it will be his time. The future consequences of hitting the proverbial robot are too great.

Source:ZDNET News