SK Hynix Unveiled Rack-Scale Memory Architecture
The new memory system aims to decouple resources from processors to increase AI inferencing efficiency by 2028.
Updated on Oct. 13, 2026 in Semiconductors
Live Poll
Do you believe new infrastructure technologies will successfully reduce AI operational costs?
SK Hynix has introduced the MARS rack-scale memory architecture and a High Bandwidth Flash memory system at the OCP Global Summit 2026. This hardware-level approach is designed to overcome memory constraints that currently force operators to over-provision expensive GPUs.
Why it matters
Traditional architectures are struggling with rigid scaling and resource imbalances that inflate ownership costs in AI data centers. By decoupling memory from specific processors, this system aims to shift from CPU-centric design toward a more scalable, rack-wide memory pool.
The High Bandwidth Flash system offers 14 times the capacity of High Bandwidth Memory (HBM) at half the bandwidth. This architecture yields a 5 times improvement in Time To First Token (the latency required to generate the first character of an AI response) for inferencing workloads.
The players
SK Hynix
A leading semiconductor supplier focused on memory technologies including DRAM, HBM, and NAND flash storage.
Hoshik Kim
Senior Vice President at SK Hynix who presented the memory architecture and roadmap.
The details
MARS—a rack-scale shared memory architecture—decouples memory modules from specific CPU or GPU server constraints to create a unified, shared memory pool accessible across a rack. High Bandwidth Flash (HBF) acts as the storage medium, providing a capacity density that HBM—a specialized memory interface using stacked DRAM dies—cannot match. By distributing this memory at the rack level, the system reduces the need for GPU over-provisioning, which is currently used to compensate for local memory bottlenecks.
Timeline
July 2026: SK Hynix released the OCP HBF specification version 0.7.
October 2026: SK Hynix introduced the MARS architecture at the OCP Global Summit.
Early 2027: SK Hynix expects to release the HBF specification version 1.0.
Early 2028: SK Hynix plans to provide HBF Cube samples.
The Tech Race
This effort follows the Open Compute Project standards to standardize memory access across hyperscale server racks. It represents a significant attempt to challenge the current dominance of local, GPU-attached HBM by creating a high-capacity, shared memory tier.
Data center operators and cloud service providers are the primary users, as this hardware aims to reduce the overhead costs associated with GPU over-provisioning. Widespread availability remains pending, with initial sampling of the HBF Cube units not expected until 2028.
The takeaway
The architecture highlights a strategic pivot toward rack-level memory sharing to address AI inferencing bottlenecks. Watch for the HBF specification 1.0 release in early 2027, which will signal the transition from research-stage specifications to final hardware design standards.
Further reading
For more on evolving memory interfaces, visit Semiconductors.
Source note: This article includes information reported by TrendForce.
Live Poll
Do you believe new infrastructure technologies will successfully reduce AI operational costs?
