OpenAI Jalapeño ASIC Beats Nvidia GB300 in Power and Latency

What is the Jalapeño ASIC?

OpenAI has unveiled its first in‑house application specific integrated circuit, dubbed the Jalapeño ASIC. Designed for high‑performance inference workloads, the chip operates at 700 watts and targets large language model serving. The architecture is co developed with Broadcom, leveraging the latter's expertise in high‑speed interconnects and silicon packaging.

Benchmark results at Hot Chips

During the Hot Chips conference, OpenAI presented a set of performance measurements that compare the new ASIC against Nvidia's flagship GB300 GPU, a 1,400‑watt accelerator widely used in data centers. The key figures were:

1. Throughput per kilowatt: 1.9 times higher on the Jalapeño ASIC.
2. Latency: 3.6 times lower than the GB300 for comparable model sizes.
3. Peak power consumption: half of the GPU while delivering more work per second.

The results were displayed on a slide deck posted on OpenAI’s official blog, and the raw data files are available for download.

Methodology

Both devices were evaluated using identical model weights and batch sizes. Power draw was measured at the wall socket with a calibrated meter. Latency numbers represent the 99th percentile of response times for a steady stream of requests.

Power efficiency compared with Nvidia GB300

The most striking difference lies in energy usage. While the GB300 consumes 1,400 watts at full load, the Jalapeño ASIC delivers nearly double the compute per watt. This translates into lower operating costs for cloud providers that run inference services at scale.

Energy‑aware operators can calculate the savings with a simple formula:

• Energy cost per inference = (Power consumption × Electricity price) / Throughput.
• Because the ASIC halves the power draw and nearly doubles throughput, the cost per inference drops by roughly 70 percent.

Thermal considerations

Reduced heat output also eases cooling requirements. Data centers often allocate a significant portion of their budget to HVAC systems. A chip that runs cooler can be packed more densely, increasing overall rack utilization.

Implications for data center workloads

Enterprises that host large language models stand to benefit from the efficiency gains. Lower latency improves user experience for interactive applications such as chat services, while higher throughput supports batch processing of analytics jobs.

In addition, the smaller power envelope aligns with sustainability goals. Many cloud providers have pledged to reach carbon‑neutral operations, and hardware that reduces electricity consumption directly contributes to those targets.

Potential cost model

A typical data center might run 100 GB300 GPUs, each drawing 1,400 watts. Replacing them with an equal number of Jalapeño ASICs would cut total power draw from 140 kilowatts to 70 kilowatts. Assuming an electricity price of $0.12 per kilowatt‑hour, the annual savings exceed $120,000.

Industry response and future outlook

Analysts have noted that the chip marks a shift toward custom silicon for inference workloads. A report from the Semiconductor Industry Association highlights the growing trend of vertical integration, where companies design both software stacks and hardware to squeeze out efficiency gains.

Broadcom’s involvement signals a deeper partnership. The company’s recent press release emphasizes its role in delivering high‑speed SerDes and power‑management IP for the ASIC.

Meanwhile, Nvidia has not yet issued a formal comment, but the company’s recent earnings call referenced ongoing development of next‑generation GPUs that aim to improve performance per watt.

What comes next?

OpenAI plans to iterate on the Jalapeño design, targeting higher clock speeds and expanded memory bandwidth. The roadmap includes a 1‑teraflop version that could further narrow the gap between custom ASICs and general‑purpose GPUs.

For customers, the immediate decision revolves around whether to adopt the new silicon now or wait for broader ecosystem support. Compatibility with existing software frameworks, such as TensorFlow and PyTorch, will be a key factor.

Overall, the benchmark results suggest that custom ASICs are becoming a viable alternative to traditional GPUs for high‑throughput, low‑latency inference. As more vendors announce similar initiatives, the competitive landscape is likely to evolve rapidly.