OpenAI Jalapeño puts NVIDIA's inference margins on the clock

Does Jalapeño beat NVIDIA? On the benchmark OpenAI published, yes. Does that make it a better chip than NVIDIA's Blackwell platform? The evidence does not support that claim yet. Should NVIDIA care? Yes. Jalapeño gives OpenAI a credible way to move repeated, high-volume inference onto hardware it controls. That changes how OpenAI buys GPUs, how much pricing power NVIDIA keeps, and how expensive it is to leave CUDA. That is a narrower claim than "NVIDIA killer." It is also more interesting. Short version: Jalapeño is an inference ASIC co-developed by OpenAI and Broadcom. Early results show excellent latency and performance per watt on three large models. It has not yet proved production-scale economics, long-context agent performance, or fleet reliability. Near term, it gives OpenAI capacity and negotiating power. Over time, it could take a profitable slice of inference away from merchant GPUs and weaken one part of NVIDIA's software moat. This is infrastructure analysis, not a stock call. What exactly is Jalapeño? OpenAI calls Jalapeño its first "Intelligence Processor." The plainer description is a custom ASIC for large-language-model inference, built with Broadcom and turned into boards, racks, and production systems with Celestica. This is intended to become more than a lab project. OpenAI and Broadcom announced a 10-gigawatt custom-accelerator program in October 2025, with racks targeted to start deploying in the second half of 2026 and the program running through 2029. Those gigawatts are a roadmap, not deployed capacity. The original collaboration announcement states the schedule. Inference is the part that happens after training. A model has already learned its weights. The system now has to process a prompt, generate tokens, maintain the conversation state, route requests, and repeat that work for millions of users and agents. NVIDIA GPUs can train models and serve them. Jalapeño has a smaller job description. It is designed around serving current and future LLMs, with special attention to memory movement, network delays, and the changing mix of prompt processing and token generation. OpenAI says the first engineering samples are running at target frequency and power, and that initial deployment is planned by the end of 2026. OpenAI's launch announcement has the full scope. That narrower job lets a purpose-built chip skip features required by a merchant platform. It can spend more of its transistor, memory, and power budget on the workload its owner runs every day. Why inference hardware keeps waiting An LLM request has two broad phases. During prefill, the system reads the prompt and builds the initial state. This phase leans heavily on compute. During decode, the model emits tokens one at a time. Decode often waits on memory bandwidth because the system repeatedly reads weights and the key-value cache, or KV cache, that holds attention state from earlier tokens. The arithmetic units can be fast and still sit idle. Data may have to cross a cache hierarchy, move between chips, or travel from a prefill worker to a separate decode worker. Every move costs energy. Some moves also add directly to the delay a user feels. Jalapeño attacks that waiting time in several ways: - Its cores and HBM are organized into local slices. Software can place weights and KV state close to the compute that needs them. - A dedicated collective network handles known high-bandwidth communication between slices. - The design supports smaller matrix shapes, which helps avoid the padding and utilization cliffs that can appear on large systolic arrays. - OpenAI uses one flexible accelerator pool instead of permanently splitting hardware into separate prefill and decode fleets. That last choice is easy to underestimate. Separate prefill and decode pools can look efficient when traffic matches the ratio used during planning. Production traffic moves. Prompt length, output length, cache-hit rate, concurrency, and speculative-decoding acceptance all change through the day. A fixed split can leave one pool idle while the other queues. OpenAI accepts some local inefficiency to keep every device available for the next request. It also avoids shipping a growing KV cache across the network between phases. SemiAnalysis' architectural analysis describes a reticle-sized compute die, HBM4 with 15.4 TB/s of bandwidth, and a system that can connect 2,048 accelerators across 16 racks. This is hardware and workload co-design in the literal sense. The team started with the traffic pattern, then built the chip and serving system around it. What did the benchmark show? OpenAI tested Jalapeño on SemiAnalysis' public InferenceX benchmark using three open models: - GPT-OSS 120B - DeepSeek R1 670B - Kimi K2.5 1T Across those models, OpenAI reports 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the comparison systems. The chip is rated at 700 watts, while measured sustained power stayed at or below 550 watts during the tested workloads. On Kimi K2.5, OpenAI reports about 1.5 times higher peak performance per watt and 3.4 times lower end-to-end latency. The results and appendix are public. The curve is more informative than the peak number. Jalapeño remained strong at lower concurrency, where each user expects tokens quickly. Coding agents care about that region because delays compound across sequential model calls. The chart stops before production evidence begins. SemiAnalysis observed the runs in OpenAI's lab, but OpenAI supplied the numbers. SemiAnalysis did not run the complete InferenceX suite and has not published AgentX results for Jalapeño. The current tests use an 8,000-token input and 1,000-token output workload. That is easier to tune than a long, multi-turn agent session with shared prefixes, cache reuse, pauses, routers, and parallel subagents. AgentX was built for that messier traffic. Until Jalapeño runs it, the published chart says less about a full Codex session than the headline suggests. There is also a timing mismatch. The benchmark compares engineering silicon against commercially available systems such as Blackwell. SemiAnalysis argues that HBM4-based Jalapeño is better compared with Vera Rubin, which is entering production earlier than Jalapeño's expected volume ramp. A0 silicon produced the current Jalapeño data. Production cost, yield, uptime, repair behavior, and sustained fleet performance remain unknown. So yes, the result is technically credible. It is not a final verdict on NVIDIA hardware. The AI-designed-chip story is more important than the branding OpenAI says its models helped the team move from design to tapeout in nine months. AI explored implementations, shortened verification loops, and optimized arithmetic circuits. The team then used Codex with an internal model to bring three unplanned open-weight models to high performance in two months. A selected-block result hints at why bring-up moved quickly. On GPT-OSS attention and mixture-of-experts blocks, OpenAI says AI-generated implementations ran 1.5 to 1.8 times faster than existing code written by human experts. OpenAI is explicit that this result covers selected blocks, not the complete model. That limit does not make the result trivial. CUDA's advantage includes mature libraries, tooling, kernels, profilers, and a huge developer base. A new accelerator normally pays a large software tax before its theoretical hardware performance becomes useful. OpenAI's early bring-up suggests that models can reduce part of that tax by writing and tuning kernels against a predictable architecture. The chip matters. The nine-month loop matters more. If OpenAI can repeat that loop, each stable and expensive workload becomes a candidate for first-party silicon sooner than NVIDIA would like. Four ways Jalapeño can hurt NVIDIA 1. OpenAI gains pricing power immediately Jalapeño does not need to replace a single NVIDIA rack to affect a negotiation. Working silicon gives OpenAI an alternative for the next block of inference capacity. NVIDIA must now compete against the cost OpenAI expects from its own hardware as well as AMD, Google, Amazon, and GPU clouds. OpenAI describes Jalapeño as a "credible first-party path" that lets it direct demand toward the best performance per dollar and maintain pricing discipline across suppliers. That language appears in OpenAI CFO Sarah Friar's infrastructure strategy. 2. The highest-volume inference can move off merchant GPUs Custom silicon works best when the owner understands the workload and can keep the hardware busy. OpenAI has both conditions. It sees the real distribution of prompts, models, cache hits, latency targets, and product demand. It can tune the full system without waiting for a merchant roadmap. My expectation is a split rather than a clean migration. OpenAI can place stable, repeated, power-sensitive inference on Jalapeño while keeping frontier training, post-training, new model bring-up, overflow capacity, and workloads that need broader software support on NVIDIA and other platforms. GPU revenue can keep growing while NVIDIA receives a smaller share of the tokens OpenAI serves. Those two outcomes can coexist because total AI demand is still expanding. 3. NVIDIA's margin is exposed before its revenue is NVIDIA reported $96.2 billion in second-quarter fiscal 2027 revenue, including $89.0 billion from data center, with a 75 percent gross margin. It also disclosed meaningful customer concentration. One direct customer represented 16 percent of quarterly revenue, while three direct customers represented 16, 15, and 13 percent of first-half revenue. NVIDIA did not identify those customers in the filing. The figures are in NVIDIA's latest 10-Q. That margin leaves room for a customer-owned chip to create savings even after Broadcom, HBM, packaging, networking, and system costs. Jalapeño can pressure NVIDIA's price before it displaces large unit volumes. NVIDIA has already writte