New WEKA Storage Boosts GPU Capacity by 7x
The technology offloads KV cache to NVMe storage to improve utilization and allow sessions to move between GPU servers.
Updated on Oct. 16, 2026 in Data Centers
Live Poll
Is now a good time to bring more AI infrastructure management in-house for your organization?
WEKA has released a new storage technology designed to offload GPU key-value (KV) cache to NVMe flash memory. This architectural shift addresses memory bottlenecks that currently force GPUs to spend significant compute cycles recomputing context instead of processing new work.
Why it matters
By moving KV cache to centralized storage, data centers can significantly improve hardware efficiency and avoid the budget and power constraints inherent in over-provisioning server DRAM. This approach enables session migration across different GPU hosts, maximizing the utility of increasingly expensive AI infrastructure.
WEKA's Augmented Memory Grid improves GPU utilization from 30% to 95% by offloading KV cache, which otherwise accounts for up to 60% of compute cycles wasted on recomputing context. This centralizes state management, effectively decoupling context storage from specific GPU memory limits.
The players
WEKA
A data platform company specializing in high-performance storage software designed for AI and GPU-accelerated computing workloads.
The details
The Augmented Memory Grid functions by intercepting KV cache—data representing the history of tokens generated in a language model session—and storing it in centralized NVMe flash memory. When a specific GPU host requires that history, the system retrieves it from the central store, enabling session resumption on any node in the data center. This architecture circumvents traditional memory limits that force systems to evict and then recompute older context, a process that severely hampers throughput on standard GPU clusters.
Timeline
October 16, 2026: WEKA announces its new storage architecture.
December 2026: Expected deadline for Google Cloud GPU reservations to avoid capacity constraints.
The Tech Race
As organizations prioritize infrastructure spending, the race to maximize GPU utilization has become the primary metric for data center performance. WEKA's approach shifts the bottleneck from raw GPU quantity to intelligent memory management, challenging standard assumptions about localized storage.
Data center operators can increase the number of users supported by a single GPU by 4 to 5 times, effectively reducing the hardware footprint per session. Organizations should prepare for restricted on-demand access to cloud GPU resources by December 2026.
The takeaway
The industry is moving toward decoupling stateful memory from compute nodes to squeeze more performance out of existing hardware deployments. Monitor upcoming enterprise capacity reports for shifts in GPU reservation availability after the December 2026 deadline.
Further reading
For more on the infrastructure challenges currently facing the industry, visit the Data Centers section.
Source note: This article includes information reported by VentureBeat.
Live Poll
Is now a good time to bring more AI infrastructure management in-house for your organization?
