Groq to Use 3D Stacked DRAM in Next-Generation Chips
The firm will shift away from SRAM to increase memory capacity for large-scale AI models.
Updated on Oct. 21, 2026 in Semiconductors
Live Poll
Do you believe larger storage capacity is more critical for AI development than faster processing speeds?
Groq plans to integrate 3D stacked DRAM into its next-generation chips to increase memory capacity. This design shift aims to support larger AI models by trading off some bandwidth speeds for significantly more memory.
Why it matters
As AI models scale, memory capacity becomes a bottleneck, making the transition from SRAM to higher-density DRAM essential for future hardware efficiency. This shift marks a strategic pivot for Groq's chip architecture.
Current Groq LPUs utilize 500 MB of SRAM and reach 150 TB/s of bandwidth, with rack-scale systems delivering 128 GB of SRAM and 640 TB/s bandwidth. Next-generation designs will use Through-Silicon-Vias to vertically stack DRAM, shifting from the current SRAM-heavy architecture.
The players
Groq
An NVIDIA-acquired firm specializing in hardware for AI inference, known for its LPUs and high-bandwidth LPU rack systems.
The details
Through-Silicon-Vias (TSVs) are vertical electrical connections that pass completely through a silicon wafer or die. By using TSVs, Groq can stack multiple layers of DRAM memory directly on top of the processor, allowing for much higher memory density than the current SRAM, a type of volatile memory that is faster but occupies more physical space on a chip. This architecture seeks to accommodate larger AI models that exceed the footprint of current on-chip memory.
Timeline
August 2026: Groq LPX racks entered full production.
October 21, 2026: The forecast regarding next-generation chip architecture was published.
The Tech Race
This transition moves Groq away from its current SRAM-centric architecture which emphasizes raw bandwidth. It places the firm on a trajectory to resolve the memory capacity limits that currently constrain the scale of AI models running on its hardware.
Developers and AI researchers currently using the existing LPX racks, which provide 128 GB of SRAM capacity, will eventually see hardware capable of handling larger models. No specific release dates for the next-generation hardware have been provided.
The takeaway
The move toward 3D stacked DRAM highlights the ongoing trade-off between memory speed and capacity in the race to support larger LLMs. Observers should watch for future technical disclosure or performance benchmarks regarding these next-generation chips.
Further reading
Explore the evolving landscape of high-performance processing in Semiconductors.
Live Poll
Do you believe larger storage capacity is more critical for AI development than faster processing speeds?
