Tech Visionary Says the Big AI Labs Don’t Get What People Want

Tim O’Reilly’s yardstick for measuring the worth of a company, person, or society has long been create more value than you capture. It’s no surprise that O’Reilly-publisher, internet pioneer, VC, conference organizer, and dispenser of tech wisdom-is applying that metric to the way people design and use AI. Specifically, he’s pushing for a future where open-source AI is an elixir for the masses. He worries that, like Microsoft in the 1990s, today’s hyperscalers are trying to lock users into their products. So he is promoting efforts to open-source AI technology-not only making critical technical details such as neural-net weights accessible, but unlocking the whole stack of an AI system, giving control to designers and users. O’Reilly sees AI as a new creative medium, which he uses extensively-and even has a blog about his chats with it. During our conversation, we discovered that we disagree about AI’s role in producing original content. Guess who took which side. STEVEN LEVY: You are all in on open-source AI. Make your case. TIM O'REILLY: First, let's make sure we're talking about the same thing. When most people talk about open-source AI, they're really just talking about open-weight models. It’s much bigger than that. In the ’90s, when everybody else was focused on open-source licenses, I was like, “No, no, it's about the architecture of the system. Does it enable participation?” Why is that needed? The big labs are reading the future wrong. They have told themselves a narrative where having the biggest, best model is the key to the future. Big models like Claude are optimizing for particular use cases, but they aren't necessarily the use cases that people want. I want to embed my own special sauce. The most important thing is having a clean separation between the model, the harness, and the application. And right now we're not getting that. They built an architecture of control rather than an architecture of freedom and participation, so they have the ability to track you. Isn’t it against the interests of big companies to give up control? Oh, it's totally against their interests. But that doesn't mean that they're making the right strategic decision. For a long time, the latest and greatest models were really better for everything. Now they’re better for some things and worse for others. People are talking a lot about how Fable and Sol are worse writers than the lower-level models. [Note: Anthropic and OpenAI would disagree.] The breakthroughs that we're getting in so-called frontier AI are actually pushing us further away from what ordinary people are going to need. We could win frontier AI here in the US, and China will kick our ass because they have lower-level models diffused widely through society. The goal is to give people the ability to innovate freely, to paint outside the lines. People worry that open-source software can be a security risk, since bad actors will be able to jump the frontier model’s guardrails. All of the cybersecurity incidents we’ve seen are from the frontier models. So risks like cybersecurity and the ability to develop pathogens are actually an argument for slowing down the frontier models more than an argument for restricting open-weight models. So you feel that with open source the AI powers will no longer be dominant? I don't predict the future. but I will say the world is proceeding the way that I hoped it would. What if the big frontier models end up like mainframes or supercomputers-aimed at really hard problems, which are not actually the thing that gets diffused throughout society? People are building things like Pi, an open-source [agentic] harness. One of the things we're working on at my nonprofit, the AI Disclosures Project, is the idea of an open-memory consortium. Mark Zuckerberg’s thesis is, he’s going to lock you in because Meta will give you the AI that knows you best. The open-source vision needs to say no to this. Open source can give you the ability to switch models, switch providers, and maintain all the context that it needs. You recently wrote a piece in the Economist about how Elon Musk-as well as other tech leaders-have so much power in their companies that they are anti-capitalist, in the sense that their desires, rather than the marketplace, determine strategy. In general, Silicon Valley has become anti-capitalist. They use the language of markets. But around 2010, the whole model came into play with Uber and Lyft. Funded by VCs, they literally spent billions of dollars to subsidize rides. The venture capitalists picked the winner rather than the market. That became the template. All the capital piles on and chokes out other possible paths. In AI, all the money is funneling into a few companies, but none seem to have a lock. Because it's a failed model. The real action is open source. Just like in the early 1990s, when everybody thought it was all a battle for who ruled the PC, and then the web came out of left field while nobody was paying attention. I think that the real AI future will be a ferment of innovation that's not being funded by venture capital-in the same way that the web wasn't funded by VCs. One of those disruptions might be your own company, built on books and writing. The book business has been in decline for 25 years. At our peak, our book business was maybe $70 million, and it's now $30 million. As books generated less revenue, we had to find a way to compensate people for sharing their knowledge, because that's our fundamental business. So now AI comes along and hoovers everything up. Can we build a set of tools that help people with that? Being able to invoke the knowledge of experts is a superpower, and we have to figure out what that looks like now. When I edit this interview I will boil down our long conversation to a shorter one, but I won’t use AI to do it. And of course, the writing in the introduction will be all mine. It’s actually our policy. I think that's a legacy thing that's going to go. My ability to have a thought partner in AI when I'm writing is very powerful. AI does your writing for you? No, no. I use it for brainstorming, and sort of just functional writing. When I interview somebody for an hour, I go to AI and say, “Turn it into something I can use.” I don't have the time to write up all those interviews. Aren’t you worried that AI will eventually supplant human writing? AI is literally a medium, in the same way that written words are a medium, or paint or music is a medium. When I talk to an AI or design art with it, I get something different out of it than you do, in the same way that Michelangelo and Van Gogh got completely different things out of paint. The idea that you can't write with AI will seem as curious as the idea that you can't make a good portrait or landscape with a camera. There will be people who are masters at expressing their ideas by summoning words from LLMs. People used to ride horses. Now, few do. We're in the early days. This is an edition of Steven Levy’s Backchannel newsletter. Read previous newsletters here. Comments Back to top