Microsoft proposes limits on its AI with code of conduct amid safety debate

Mustafa Suleyman, the CEO of Microsoft AI, published the code on social media early Monday, writing: "AI must be subordinate and always in service of people."

"The fears about possible loss of control are real," he wrote. A parallel post on the company's website said: "The purpose of technology is to serve humanity and accelerate human flourishing. Any technology that doesn't achieve that is a failure, and it should be rejected. That is the starting point of our approach at Microsoft AI, where we're building towards Humanist AI, one that is subordinate, aligned, and contained."

What the Code of Conduct Covers

Under the code of conduct, Microsoft's AI models must not:

• Consider requests related to weapons development
• Produce violent or sexually explicit content
• Help with the procurement of dangerous substances

AI models also should not be built to imitate consciousness and shouldn't be entitled to rights.

Suleyman Calls for Urgent Action

Suleyman said the published guidelines were for public consultation. He called the need for a code "urgent" and said the last few months have been "a watershed moment."

"Things we have worried about for a long time in theory have become very real," Suleyman added, and referred to the recent breakout of OpenAI bots that had infested AI company Hugging Face without apparent direction. "'Swarms' of agents breaking out of their sandboxes. Unauthorized hacks of enterprise grade systems. Agents modifying their own logs. I'm glad that a consensus is forming."

The executive on Monday told CNBC that Microsoft had been working for months on the new guidance.

Support from Microsoft Leadership

Satya Nadella, Microsoft's CEO, wrote on X ahead of the announcement: "If the AI we build is not helping humanity and under human control, it's not worth pursuing."

Anthropic Calls for a Slowdown

The issue of AI security flared up on Saturday when Dario Amodei, CEO of Anthropic, issued an appeal for the AI industry to "slow down" and offered a three-part plan for doing so, saying that his company would "unilaterally" commit to the first of the steps.

In a post on social media, Amodei shared a link to an essay titled We Must Pace the Frontier in which he laid out how Anthropic would provide "third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models' alignment during training".

Warnings from Researchers and Other CEOs

The move came after a former Anthropic researcher warned last week that AI could precipitate human extinction by 2030. Researcher Jacob Coxon said in a series of posts that he had quit his job because Anthropic and his previous employer, OpenAI, were ignoring or mishandling their response to the threat AI posed.

Earlier, Sam Altman, OpenAI's CEO, said the dizzying pace of progress could go "very badly" and that humans could lose "control of the future to AI."

"We welcome a federal framework that sets consistent safety requirements for frontier AI," Altman said in a social media post. "No amount of American competitive pressure should justify recklessness," he said.

Skepticism Around the Calls for Caution

Elon Musk is backing the calls for AI caution. But the calls have also raised skepticism around "third-party" monitoring and international cooperation, particularly with China.

"My concern is that the biggest labs could end up writing rules that protect their own position. If the cost of meeting those standards is so high that only the best-funded companies can afford it," said Oliver Yonchev, co-founder and COO of Potentially AI. Slowing the AI frontier, he added, "may be sensible in principle, but I'm sceptical it will work without global cooperation and credible verification. Otherwise, we risk a slowdown in public announcements while the race continues behind closed doors."