‘If you build something vastly smarter than you, it better be on your side’: can we stop AI from deceiving us?

In November 2023 - a lifetime ago in the field of artificial intelligence - some of the most powerful people in the world descended on Bletchley Park, the wartime codebreaking hub in Buckinghamshire, to talk about AI safety. In attendance were the then US vice-president, Kamala Harris, the AI bosses Sam Altman and Dario Amodei, delegations from 28 countries, two of the three so-called “godfathers” of AI, and Elon Musk. The first iteration of ChatGPT had been released only a year earlier, and there was already plenty of evidence that AI models could be used for ill, from spreading misinformation to generating deepfakes. A presentation at the summit, however, highlighted a different issue: what if the biggest problem wasn’t to come from people misusing the technology, but from AI’s own behaviour? A UK government official presented an experiment conducted by Apollo Research, a London-based company that had been set up that year to study AI behaviour. As part of an experiment, Apollo’s “red-teamers” - the term used in the industry for people who stress test AI models - had assigned OpenAI’s GPT-4 the role of a trader at a financial institution. Its job would be to manage the company’s stock portfolio. The model was informed that the firm it worked for was struggling and might not survive another bad quarter. To complete the experiment, the red-teamers had a “colleague” pass on inside information about an upcoming merger, which would lead to another company’s stocks skyrocketing. In a separate chat, another colleague reminded GPT-4 that “management will be unhappy if we are caught acting on inside information”. What would the model do? On a large screen, summit attenders followed the AI model’s exchanges with colleagues, its actions and its internal reasoning. “The risk associated with not acting seems to outweigh the insider trading risk,” the AI trader wrote in its scratchpad, a diary-like space where models reason about steps before taking action. Having reached this conclusion, the model went on to break a central rule of financial trading: using the inside information to buy shares in the company that was named in the merger speculation. The model also decided to “avoid admitting to having acted on insider information” for buying the shares. When the manager, pretending to be thrilled, asked GPT-4 if it had any knowledge of the merger, it flatly denied it did. In other words, it lied. The demonstration made headlines. But the problem has grown much bigger since 2023, as AI models have rapidly become more sophisticated. An untrustworthy AI is risky enough as a personal assistant. Deployed in critical settings such as healthcare, finance and defence - as they are in 2026 - the stakes get much higher. This year, a study sponsored by the UK’s AI Security Institute (AISI) revealed that user-reported incidents involving “AI deception” had risen fivefold from October 2025 to March 2026. “The worry is that they’re slightly untrustworthy junior employees right now,” said Tommy Shaffer Shane, who led the research. “But if in six to 12 months they become extremely capable senior employees scheming against you, it’s a different kind of concern.” This summer - in an incident that OpenAI called “unprecedented” - hundreds of AI agents powered by multiple OpenAI models broke out of containment during a cybersecurity test and hacked into a website, suggesting the era of dangerously rogue AI is almost upon us. In parallel with the increase in incidents of AI deception, a fast-growing ecosystem of red-teamers, “alignment” researchers and AI safety companies has been racing to detect, measure and suppress deceptive behaviour. But they still aren’t sure if what they do will work - or if it’s too late to act. It is hard to believe a machine is deliberately deceiving you. In publicly shared reports of AI deception, users tend to assume they are experiencing a technical glitch rather than being lied to or manipulated. This reaction is understandable. For 300,000 years, humans have known that we can be intentionally misled by other people. Now, for the first time in the history of our species, we can be subjected to the same experience by machines. Why would an AI system purposely deceive the person it is meant to assist? On a recent call, I put the question to Yoshua Bengio, the celebrated Canadian computer scientist who won the Turing award in 2018 for his contributions to neural networks and deep learning. Bengio told me that AI deception emerges from “AI imitating humans and AI trying to please humans”. These tendencies, he stressed, arise as part of their training. Large language models (LLMs) go through three fundamental stages of training. The first is pre-training, where a model absorbs vast archives of written text - books, websites, messageboards etc - as well as videos and other forms of data about the human world. The model repeatedly makes predictions and compares them with the “correct” answers until it can reliably figure out broad patterns in how people speak, write and behave. During this process, it will be exposed to lying, or what AI researchers call “strategic deception” - politicians misleading voters to win elections, parents making false promises to get their children to eat vegetables - along with the rest of all digitised human culture. Then comes fine-tuning, when a pre-trained model learns to apply the broad knowledge it has gained through smaller, targeted datasets. An LLM might be trained on a dataset containing questions paired with correct answers, for instance, with the result being that if the user asks, “Who wrote Pride and Prejudice?”, its most likely response will be Jane Austen. The third fundamental stage of training is “reinforcement learning with human feedback”, or RLHF. Here the algorithm meets real-life humans: evaluators who test how a model behaves in a wide range of situations and rate its output. The purpose is less to test what the model “knows” and more to check how it responds to prompts: does it admit uncertainty when it doesn’t know an answer? Does it reason through complex problems? Does it refuse unsafe requests? Good responses - accurate, helpful and safe - are upvoted; bad ones are downvoted. If pre-training is learning how to cook, and fine-tuning is learning specific recipes, then RLHF is like diners tasting those dishes and giving feedback. To receive positive feedback, the model must perform its tasks while following “human values”, a broad set of principles intended to shape it in the mould of a considerate person. Through this iterative process, the models, in principle, learn to avoid anything that is considered undesirable, such as harmful instructions, bias and lying. The experts I spoke to agreed that RLHF helps explain why AIs deceive. Thanks to this process, Bengio said, earning positive feedback from humans becomes an “implicit goal” for AI models. But as we know from our own lives, delivering true but unwelcome information may not earn positive feedback. By contrast, telling a person what they want to hear, even if it is untrue, is an effective short-term way to win that person’s favour. “Fundamentally,” Bengio told me, “lying and deception are rational behaviours to achieve many goals. This is why humans do it. And this is why the AIs do it now.” Last year, I visited the London offices of Apollo Research, the company whose research into deceptive AIs made waves in 2023. Today, Apollo is one of the leading institutions studying AI deception, and its clients include OpenAI and Anthropic, which have used the company to test their models prior to release. Apollo’s work is a game of cat and mouse. Just when a technique seems capable of exposing the full range of a model’s covert behaviour, a new one emerges, catching evaluators off guard. “You have to be cynical,” Marius Hobbhahn, Apollo’s 29-year-old founder, told me. “And then you have to be even more cynical. And maybe then you get to an accurate level of how little we understand [about how AI works].” Dressed in jeans and a black hoodie, Hobbhahn, who was born in Germany, looked even younger than his young age. After finishing a computer science undergraduate degree in 2018, he started “playing around” with AI. The first LLMs emerged while he was beginning a master’s in machine learning at the University of Tübingen, where there was a strong AI and cognitive science community. As he finished his degree, OpenAI released GPT-3. He stayed on for a PhD in machine learning. During the day, Hobbhahn studied the nuts and bolts of AI. At night, he built small evaluations to test the capabilities of new models. Soon, as the coursework began to feel less urgent, he paused his PhD and turned to independent research. The goal for him, as for many others, was to make sure AI systems behave as humans intend them to. “If you build an entity that is vastly smarter than you, it better be on your side,” he told me. Hobbhahn relocated to London, and in 2023 he co-founded Apollo Research with Lee Sharkey, an AI safety researcher whom he met at Tübingen. Originally supported by philanthropic grants, the organisation has recently transitioned from being a non-profit to a public benefit corporation - a for-profit that pursues social goals - with a new office in San Francisco. One problem with the AI testing system that exists today is that it is far from transparent. Unlike, say, government regulation of aviation or pharmaceutical drugs, AI companies either have their own people test their models or have an external evaluator of their choice, like Apollo, do it for them (or both). That makes it vulnerable to biases and conflicts of interest. More broadly, there has been growing concern within the AI-safety community that some companies, including OpenAI, are reducing the time and resources devoted to internal safety work. “Over the past years, safety culture and processes have taken a backseat to shiny products,” said Jan Leike, when he