AI systems don’t have a drive to survive. Here’s why

In controlled safety tests described earlier this year, researchers asked AI models to solve a series of simple math problems. Partway through the exercise, the instructors warned the bots that if they tried to solve the next problem, the computer environment they were operating in would be shut down. In some runs, the shutdown occurred as specified. In others, models interfered with the shutdown script and continued with the remaining problems. What explains this behavior? Do artificial intelligence systems have a drive to survive? Resistance to instructions Consider a robot vacuum cleaner. When it detects that its battery is running low, it returns to its charger, recharges itself, then resumes cleaning. The machine is keeping itself going, but no one is panicking about an existential Roomba threat to humankind because the behavior is straightforward. Designers built in the ability to recharge. Artificial intelligence models are more complex. For years, researchers have discussed scenarios in which an AI agent might act to prevent shutdown so it could complete the tasks it had been assigned. Staying “on” could be a means to an end, even without an explicit instruction to resist shutdown. One recent experiment showed that when researchers spelled out to an AI agent that allowing a shutdown took priority over completing a task, the resistance disappeared. But another, much broader experiment found some resistance even when the researchers instructed the models that allowing a shutdown had priority. Why some resistance remained is an open question. What evolution explains Self-protective behavior in an AI model may look like a drive to survive. The historian Yuval Noah Harari argued in a recent interview with The Economist that “the first thing that basically any entity learns as it develops is to survive.” He also said that evolution drives organisms toward survival. In a different article, he called survival “the most basic goal of any agent.” But does invoking evolution explain why machines or even living things behave in self-preserving ways? A rabbit running from a fox is trying to save itself, and this kind of behavior has likely been favored in evolution. But natural selection is a historical process through which characteristics that contribute to survival and reproduction become more common across generations. It does not give an individual rabbit an instruction to “stay alive.”