OpenAI’s first published tests show Jalapeño, its first custom inference chip, outperforming NVIDIA Blackwell systems on speed and power efficiency. That gives OpenAI a new choice: which workloads should run on its own silicon. Across GPT‑OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T, OpenAI reported 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency. For first-generation hardware built to run large AI models rather than train them, that is a striking start.

That work is called inference: the computation that happens every time a trained model produces an answer. It becomes especially demanding for AI agents, which may call a model repeatedly as they work through a task, allowing small delays and costs to compound across many steps. As usage grows, buying more chips alone does not remove the physical constraint. Data-center power and infrastructure cannot expand as quickly as the demand placed on them, so OpenAI needs each available watt to produce more useful AI without making users wait longer.

Jalapeño is OpenAI’s attempt to close that gap. The company designed the chip, memory, network, and serving software around the way large models actually run, then used its own AI to help design and program the system. If the early performance survives production deployment, businesses and developers using OpenAI’s products and APIs could get faster agents and more dependable capacity, while OpenAI gains greater control over its serving costs and where suitable workloads run. For NVIDIA and OpenAI’s other infrastructure suppliers, Jalapeño does not have to replace their hardware to matter: a credible first-party inference option could change how OpenAI distributes workloads as demand grows. The benchmark results are the first evidence that this strategy may work.

OpenAI measured Jalapeño against systems using NVIDIA’s GB200 and GB300 chips by asking two practical questions: How much AI work could each system complete for every watt of electrical power its chips were rated to use, and how long did each request take from beginning to end? The first measure, called throughput, shows how much demand a system can handle. The second, called latency, shows how long each user waits. Those goals usually pull against each other because processing more requests can make individual answers slower. That is what makes Jalapeño’s early result striking: in OpenAI’s tests, it handled more demand per watt and returned answers faster.

OpenAI conducted the tests using InferenceX, a public benchmark created by SemiAnalysis. During the tests, OpenAI sent each system requests containing approximately 8,000 input tokens—the pieces of text sent to the model—and required each model to generate approximately 1,000 output tokens per request. Reading the input relies heavily on computation. Producing the answer token by token requires the hardware to retrieve model information from memory repeatedly. Because those two parts of a request stress the hardware differently, InferenceX measures both.

InferenceX measured the complete setups around the chips, including their memory, networking and serving software. Software decides how models and incoming requests are divided across the chips, while memory and networking deliver the information needed to perform that work. Delays anywhere in that process can leave the chips waiting. The results therefore show how OpenAI’s Jalapeño-based setup performed against systems built around NVIDIA chips.

Jalapeño and the NVIDIA hardware have different published wattage ratings: 700 watts for Jalapeño, 1,200 watts for GB200 and 1,400 watts for GB300. OpenAI divided each system’s token-processing rate by those ratings and reported how much work it completed per kilowatt. This kept a system with a higher wattage rating from appearing more efficient simply because it could use more electrical power.

OpenAI ran each model on Jalapeño and an NVIDIA-based comparison system:

Those comparisons show both how much work each system completed per watt and how long the complete request took. The same InferenceX results also show how quickly the answer streamed once generation began:

OpenAI used those results to compare the systems another way. It matched Jalapeño to the fastest per-user response speed achieved by each NVIDIA system, then measured how much total work each could process at that equal speed. Here, total work includes both the prompt tokens being read and the response tokens being generated:

Altogether, the three comparisons show that Jalapeño completed more work per watt, returned and streamed individual answers faster, and carried substantially more total demand when the per-user response speed was held equal.

OpenAI conducted the Jalapeño tests and supplied the performance numbers. SemiAnalysis, the independent semiconductor research firm that created InferenceX, observed selected runs and confirmed in one evaluation that Jalapeño’s output quality was comparable to the NVIDIA systems. SemiAnalysis did not conduct a complete independent benchmark, and the published results do not include its longer, multi-step AgentX test, which more closely resembles how AI agents work.

SemiAnalysis also found that Jalapeño produced more output tokens per watt than early published results for Rubin, NVIDIA’s newer generation after Blackwell. The Rubin results came from different tests and configurations, so they cannot establish a direct winner, but they show that Jalapeño’s early efficiency remains competitive even against newer NVIDIA technology.

OpenAI says Jalapeño’s performance comes from designing its chip, memory, networking and software as one system around real AI workloads.

An AI request changes the kind of work the hardware must do as it moves from the prompt to the answer. During the first phase, called prefill, the system reads and processes the prompt, which requires substantial computing power. During the second phase, called decode, the model generates its answer one token at a time and repeatedly retrieves information from memory. Data also has to travel between memory, processors and chips. Each transfer can consume electricity and leave parts of the system waiting instead of producing an answer.

Jalapeño’s central design choice was to reduce that movement. OpenAI says the system can place and keep model information close to the computing resources using it. That information includes the KV cache, a working record the model relies on to remember the prompt and continue generating its response. Keeping it nearby can shorten the wait for information and reduce the power spent moving it around.

OpenAI also designed Jalapeño’s network as part of the system rather than treating it as a separate connection between chips. The network is intended to keep an entire workload within one connected system while directing the right combination of computing power, memory and networking to each phase of a request. When less time is lost moving information or waiting for it to arrive, more of the available power can go toward useful AI work. That helps explain how Jalapeño could handle more demand per watt while also returning answers faster in OpenAI’s tests.

That balance becomes especially important as AI workloads change. OpenAI says Jalapeño can activate the right combination of computing power, memory and networking for prompt processing or answer generation, instead of assigning the two phases to separate fixed pools of hardware. An agent may alternate repeatedly between compute-heavy prompt processing and memory-heavy answer generation as it works through a task. A system optimized mainly for one phase can lose speed and efficiency during the other.

Its performance across GPT‑OSS, DeepSeek and Kimi provides early evidence that this approach can work across different model families. The results do not prove that Jalapeño will perform equally well on every model or production workload, but they suggest that OpenAI has built an adaptable inference platform rather than hardware limited to one proprietary model. That flexibility still depends on software tailored to each new model. OpenAI says its own AI helped accelerate that programming, which helps explain how its first-generation chip became competitive so quickly.

OpenAI began using AI before Jalapeño reached the factory. The company says it moved from the chip’s initial design to tapeout—the point when a completed design is sent for manufacturing—in nine months. Its AI models helped engineers explore possible implementations, test and verify their work more quickly, and improve the chip’s arithmetic circuits. Shortening those repeated design loops helped OpenAI fit more computing performance into Jalapeño while keeping the project on schedule.

OpenAI also had to write software that would let Jalapeño run each new model efficiently. Each model family requires kernels, the low-level instructions that tell the chip how to perform specific operations. OpenAI designed Jalapeño to give both engineers and AI a predictable way to describe where information should be stored, where each piece of work should run and how operations should be coordinated. That allowed AI to search for more efficient ways to use the hardware instead of requiring engineers to evaluate every option themselves.

OpenAI says it used Codex with GPT‑Astra to bring three open-weight models that were not included in Jalapeño’s original production plan to high performance within two months. That speed matters because an inference chip becomes more useful when it can adapt to new models without requiring years of additional software development. It also helps explain how Jalapeño produced competitive results across GPT‑OSS, DeepSeek and Kimi despite being a first-generation platform.

The most striking result came from selected parts of GPT‑OSS. For attention and mixture-of-experts blocks—two types of operations used inside the model—AI-generated implementations ran 1.5 to 1.8 times faster than the existing versions written by human experts. The result applies only to those selected blocks, not the complete model or Jalapeño’s entire software system. Even with that boundary, it shows that OpenAI’s models could do more than automate routine programming: in some cases, they found faster ways to run AI workloads on the company’s own chip.

Together, these results suggest that Jalapeño’s advantage extends beyond the physical hardware. OpenAI may be able to shorten the cycle between encountering a new workload, adapting its software and improving how efficiently the chip runs it. That faster cycle could help OpenAI turn each improvement into additional inference capacity sooner, increasing the commercial value of owning more of the system that runs its products.

That faster optimization cycle matters commercially because each efficiency gain can expand the amount of AI OpenAI is able to serve. If Jalapeño produces more useful work from each watt, OpenAI can handle additional requests without waiting for an equal increase in electrical power. The company describes this as operating leverage: useful work and revenue could grow faster than the cost of serving that demand. OpenAI has not disclosed a production cost per token or independently verified savings, but the early efficiency results show how Jalapeño could create more capacity within a limited power supply.

Customers could experience that additional capacity through faster answers, more responsive agents and more dependable access when demand is high. OpenAI says the gains could allow Codex and other agents to complete more steps with less waiting while making API products less expensive to build. Those are possible uses of Jalapeño’s efficiency, not announced changes to OpenAI’s prices or products. The immediate value is giving OpenAI more room to serve growing usage without increasing power and infrastructure at the same rate.

OpenAI designed Jalapeño as an inference chip. The company can route suitable inference workloads to its own platform while continuing to use accelerators from NVIDIA and other providers for the rest of its inference demand and for training frontier models. OpenAI says that preserving choices across hardware and providers allows it to direct demand toward the strongest performance at the right cost and maintain pricing discipline as the market changes. A credible first-party chip could therefore improve both capacity planning and OpenAI’s control over the economics of serving its products.

OpenAI also relied on Broadcom and Celestica to turn Jalapeño’s design into deployable hardware. Broadcom contributed silicon implementation and networking technology, while Celestica supplied expertise in boards, racks and complete systems. Those partnerships give OpenAI more control over the chip’s architecture without requiring it to manufacture and industrialize every component itself.

OpenAI expects to begin deploying Jalapeño by the end of 2026. OpenAI hardware chief Richard Ho told TechCrunch that the initial volume would be very small, with more significant deployment expected in 2027. OpenAI has not disclosed how reliably Jalapeño will operate at scale, its production cost per token, the share of workloads it will handle or how much money the chip could save.

If its early performance survives production deployment, OpenAI could move suitable inference workloads onto its own platform, serve more demand within its available power and gain greater control over cost and capacity. That would make infrastructure another source of competitive advantage alongside the models running on it.

Jalapeño’s first published results make OpenAI’s custom-chip strategy unusually credible for a first-generation platform. In the company’s tests, it completed more inference work per watt and returned answers faster than Blackwell systems across three large model families. Producing those results with a first-generation chip is what makes Jalapeño more than another custom-silicon announcement.

OpenAI may also have found a way to reduce the usual disadvantage of starting a new chip program. The company says AI assisted Jalapeño’s hardware design and later helped bring three unplanned models to high performance within two months. On selected model blocks, AI-generated implementations even outperformed versions written by human experts. Together, those results suggest OpenAI can shorten the time between encountering a new workload and improving how efficiently its hardware runs it.

That faster improvement cycle matters because inference demand is growing faster than data-center power and infrastructure. Stronger models cannot supply the capacity required to serve them. Production deployment will determine whether Jalapeño’s early gains hold, but the results point to an unusual competitive possibility: the AI OpenAI builds could help improve the infrastructure that determines how much AI the company can deliver.

Jalapeño therefore matters less as proof that OpenAI can build a chip than as an early glimpse of something potentially more consequential: AI helping build and improve the infrastructure that will run the next generation of AI.

Q: What is OpenAI’s Jalapeño chip?
A: OpenAI’s Jalapeño is the company’s first custom inference chip, designed to run trained AI models by coordinating computing, memory, networking and serving software as one system.

Q: How does Jalapeño compare with NVIDIA Blackwell?
A: In OpenAI’s published tests, Jalapeño completed 50% to 90% more work per kilowatt and returned answers 43% to 72% sooner than NVIDIA GB200 and GB300 systems across three large model families. SemiAnalysis observed selected runs and confirmed comparable output quality in one evaluation, but it did not conduct a complete independent benchmark.

Q: Why is Jalapeño more efficient for AI inference?
A: OpenAI says Jalapeño improves inference efficiency by keeping model information closer to the computing resources using it and coordinating the chip, memory, networking and serving software around each phase of an AI request.

Q: How did OpenAI use AI to develop Jalapeño?
A: OpenAI says AI helped engineers move Jalapeño from its initial design to tapeout in nine months, while Codex with GPT‑Astra helped bring three unplanned models to high performance within two months. AI-generated implementations also ran 1.5 to 1.8 times faster than human-written versions on selected model blocks, not across the complete model.

Q: Will Jalapeño replace NVIDIA GPUs?
A: Jalapeño is not expected to replace NVIDIA hardware across OpenAI because the company can move suitable inference workloads to its own chip while continuing to use NVIDIA and other accelerators for remaining inference demand and frontier-model training.

Q: When will OpenAI deploy Jalapeño?
A: OpenAI expects Jalapeño deployment to begin in very small volumes by the end of 2026, with more significant deployment anticipated in 2027. Its reliability at scale, production cost per token, workload share and financial savings remain unknown.

Editor’s Note: This article was created by Alicia Shapiro, CMO of AiNews.com, with writing support, AEO/GEO/SEO optimization, image concept development, and editorial structuring support from ChatGPT, an AI assistant. All final editorial decisions, perspectives, and publishing choices were made by Alicia Shapiro.