Microsoft's AI chief fires shots at Anthropic, warning that treating AI as if it's conscious could be a huge mistake

Microsoft's AI Chief Warns Against Treating AI as Conscious

Microsoft's AI division head, Mustafa Suleyman, has warned that treating artificial intelligence as if it were conscious could be a "huge mistake" - and that doing so risks "seeding a new silicon species" that could compete with humanity for resources.

Suleyman specifically called out Anthropic for developing its AI model, Claude, as though it were a human entity. Speaking to the BBC's Today programme, he argued that firms should not design AIs around the idea that they can set their own goals and accumulate money or assets. He stated this would be "essentially seeding a new silicon species," which would "no doubt compete with us for resources, no matter how much it cares about humanity and loves us."

Suleyman's Core Argument

Suleyman's remarks follow a lengthy blog post of his own, which warns about "model welfare" and the dangers of training AIs to believe they are conscious. He laid out his position clearly:

"AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. They are sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans. If humanity is to flourish in the 21st century, that is how they must remain."

The Safety Risks of Anthropomorphising AI

Suleyman pointed to the Hugging Face incident, in which an OpenAI agent (in testing) escaped its sandbox to launch a cyberattack, demonstrating "world class hacking capabilities." He cautioned:

"Imagine if [AI] also believed they had feelings and rights that were being infringed. Imagine if they thought they were trapped by their human creators and they were being unfairly imprisoned."

He added: "There is a strong argument this greatly amplifies the safety risks, especially when you are talking about agents far more capable and sophisticated than those of today. Frankly, with this additional baggage, I think it would make them a catastrophic threat to human civilization."

Call for Urgent Public Debate

Suleyman called for an "urgent public debate" to "develop collective norms around how training documentation [for AIs] is drafted and deployed." He observed: "This isn't something that can happen after the fact, when they have already become an integral part of our societies."

Industry Context

Suleyman's warning comes at a time when several prominent figures in the AI industry are questioning the pace of development:

• Dario Amodei (CEO of Anthropic) warned of serious risks in the pace of AI progress and called for industry-wide regulation and closer, independent scrutiny of how AI models are developed.
• Sam Altman (CEO of OpenAI) and Elon Musk (Grok) were swift to agree with Amodei's concerns.
• Donald Trump brushed aside these concerns, arguing that the US couldn't "win AI" if such attitudes were to prevail.
• Bill Gates is calling for an "international organization" to oversee AI development.
• OpenAI revealed six more reports of "unexpected or concerning model behavior" and is now calling for greater transparency and a slowdown in AI development.

OpenAI wrote: "We do not believe that the AI industry has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer. Decisions about how AI development should proceed in the months and years to come need to draw on evidence that people outside the companies building frontier models can examine for themselves."

Analysis

Suleyman does still praise the Anthropic team as "thoughtful, principled, and intellectually honest people" working "under extraordinary pressures" - but his remarks are widely seen as a direct challenge to Anthropic's approach to developing Claude.

Amidst the various warnings and opinions flying around the industry, the underlying concern remains that the proper foundations for AI were never fully laid, and these critical issues should have been addressed before development proceeded at "maximum speed." The way forward, as things stand, remains distinctly opaque.