Groq LPU
AI inference accelerator with deterministic dataflow architecture
The Groq LPU is an AI accelerator designed for low-latency inference. Created by Jonathan Ross, it uses a static scheduling model and on-chip SRAM. Nvidia licensed the technology in late 2025 to enhance its Vera-Rubin platform.
- Uses 128 GB of on-chip SRAM per rack
- Delivers 315 PFLOPS of FP8 inference compute
- Nvidia licensed the technology in late 2025
- Designed by former Google engineer Jonathan Ross
- Third generation LP30 made by Samsung
What it is
The Groq LPU, or Language Processing Unit, is a specialised chip for artificial intelligence inference. It was originally called a Tensor Streaming Processor. The design focuses on speed and predictability rather than raw throughput alone. Groq developed the hardware to handle large language models efficiently. The company claims its racks offer significant compute power with high bandwidth memory directly on the chip. This approach differs from traditional graphics processors that rely on external high-bandwidth memory stacks.
How it came about
Jonathan Ross, a former Google engineer, created the underlying architecture. He built a fully scheduled tensor processing unit after leaving the search giant. When the generative AI boom began, the team renamed these chips Language Processing Units. The core design remained unchanged. In late December 2025, Nvidia engaged in a twenty billion dollar acquihire of most of the Groq development team. They also licensed the technology for their own platforms.
Inside the chip
The LPU uses a statically scheduled, deterministic compute engine. This means operations are planned in advance, unlike dynamic scheduling in general-purpose GPUs. It distributes model weights across aggregate static random-access memory. This memory sits on the chip itself, reducing latency. The architecture supports a small number of users per engine, often just one. Response time for token generation improves as more machines are added to the cluster. It is designed specifically for the decode phase of AI workloads.
How it was made
Samsung manufactures the third generation of these chips, known as LP30 or Alan-3. Nvidia announced plans to bring this generation to market in the second half of 2026. The previous generations were produced to support Groq's own cloud infrastructure. The manufacturing process details for the specific node are not provided in the available records. The focus remains on the integration of the chip into larger rackscale systems.
Where it went
These accelerators target production AI inference where low latency is critical. They are used in data centres operated by Groq across four continents. The technology suits agentic AI systems where many tokens must be generated quickly. Nvidia intends to integrate the LPU into its Vera-Rubin platform. This combination aims to optimise the decode step for large context windows. It serves premium tiers of service where response time directly impacts cost and utility.
Versions and family
The third generation chip is called the LP30. Internal references suggest it was also known as Alan-3. Earlier versions were referred to generally as LPUs or Tensor Streaming Processors. Nvidia plans to connect 256 of the next-generation LPX accelerators in a single rack. This configuration links to Nvidia Vera Rubin systems. The LPX variant represents the evolution of the original LPU design under Nvidia ownership.
Why it matters
The acquisition by Nvidia highlights the industry shift towards specialised inference hardware. Traditional GPU architectures face challenges with the latency demands of agentic AI. The LPU architecture offers a different balance between memory bandwidth and compute. It allows for lower cost per token in interactive scenarios. This has influenced how major cloud providers view the hierarchy of AI training versus inference hardware.
Questions people ask about Groq LPU
What does LPU stand for?
LPU stands for Language Processing Unit. It was previously known as a Tensor Streaming Processor. The name changed when generative AI became popular, but the underlying architecture remained the same.
How is the Groq LPU different from a GPU?
The LPU uses static scheduling and on-chip SRAM, whereas GPUs use dynamic scheduling and external HBM. The LPU is deterministic and optimised for low-latency inference, while GPUs are better for batching and training.
Who manufactures the Groq LPU?
Samsung manufactures the third generation of the chip, known as the LP30. Nvidia has licensed the technology and is working with Samsung to bring these chips to market for their Vera-Rubin platform.
What is the main advantage of the LPU architecture?
The main advantage is low latency for token generation. By keeping model weights in on-chip SRAM and using deterministic scheduling, it reduces delays. This makes it suitable for agentic AI where speed is critical.
Good reading and sources
There is no English Wikipedia article on Groq LPU yet, so the facts on this page come from the sources listed above: groq.com/lpu-architecture, nextplatform.com/ai/2026/03/17/nvidia-finally-admits-why-it-shelled-ou. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryChip wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.