Allen Institute Released Olmo-Core 3 Framework
The open-source framework enables training trillion-parameter mixture-of-experts models with improved throughput.
Updated on Oct. 2, 2026 in Artificial Intelligence
Live Poll
Should advanced artificial intelligence development frameworks be made freely available to the open-source community?
The Allen Institute for AI has released Olmo-core 3, a development framework designed to support the training of mixture-of-experts large language models at the trillion-parameter scale. The software is now available for developers on GitHub.
Why it matters
The framework aims to bridge the operational gap between dense models and mixture-of-experts architectures, allowing researchers to scale to one trillion parameters while managing computational costs. It addresses the growing need for efficient training methods as model complexity increases.
Olmo-core 3 demonstrated a 2.7 times throughput improvement over the Nvidia Megatron-core training architecture. The framework supports a pool of up to 128 experts, selecting four experts per token.
The players
Allen Institute for AI
A Seattle-based research organization focused on advancing artificial intelligence through open science and development tools.
Nvidia
A leading designer of specialized hardware and software stacks for high-performance AI training and computing.
The details
The architecture utilizes expert parallelism, a technique that distributes individual experts across multiple GPUs, and partitions model layers across groups of processing units. To optimize memory, a distributed optimizer spreads the training state across multiple GPUs rather than requiring full copies on every unit. Additionally, the system supports the MXFP8 number format, a low-precision data format that reduces memory usage during computation.
Timeline
October 2, 2026: The Allen Institute for AI officially announced the availability of Olmo-core 3.
The Tech Race
The development follows the competitive trajectory established by the Nvidia Megatron-core training architecture. By prioritizing throughput efficiency, the framework directly targets the hardware bottlenecks currently limiting trillion-parameter model development.
Developers and researchers can access the framework immediately via GitHub to integrate it into their own training pipelines. The system is designed to run on hardware such as Nvidia B3000 GPUs, making it a new resource for teams scaling large language models.
The takeaway
The transition to trillion-parameter models requires architectural shifts that move beyond simple dense training methods. Watch for future benchmarks comparing Olmo-core 3 against other scaling frameworks to confirm if these efficiency gains hold at higher parameter counts.
Further reading
For more on the current landscape of model training, visit the Artificial Intelligence section.
Live Poll
Should advanced artificial intelligence development frameworks be made freely available to the open-source community?
