DeepSeek Inference Cluster Hits 155 Trillion Tokens Daily
The company open-sourced its DeepInfer framework to help developers optimize model inference costs and infrastructure throughput.
Updated on Oct. 24, 2026 in Artificial Intelligence
Live Poll
Do you believe current AI service throughput levels signal actual long-term profitability for the industry?
DeepSeek recently disclosed that its inference cluster processes 155 trillion tokens daily. The company also open-sourced its DeepInfer framework, which utilizes techniques like expert parallelism and speculative decoding to drive efficiency.
Why it matters
Quantifying inference throughput and efficiency is critical for understanding the cost-at-scale of large language models. These disclosures provide a benchmark for evaluating the operational expense of high-volume AI services.
DeepSeek reports a throughput of 155 trillion tokens daily, with theoretical API billings of $38 per GPU-hour. This usage pattern implies an annualized revenue capacity exceeding $330,000 per GPU under continuous operation.
The players
DeepSeek
An AI research lab developing large language models and infrastructure, known for open-sourcing its training and inference frameworks.
The details
The DeepInfer framework achieves these efficiency gains through four primary mechanisms: expert parallelism, which distributes model weights across multiple nodes; speculative decoding, a process where a smaller model drafts text before a larger model validates it; asynchronous scheduling to reduce wait times; and KV Cache optimization to minimize the memory overhead of storing context keys and values.
Timeline
October 24, 2026: DeepSeek disclosed cluster throughput and inference efficiency metrics.
The Tech Race
This disclosure follows the industry trend of open-sourcing specialized inference stacks to lower the cost of model deployment. DeepSeek is positioning its infrastructure metrics as a performance benchmark against proprietary cloud AI service providers.
Developers and AI researchers gain access to the open-source DeepInfer framework to build or optimize their own inference workflows. The reported $38 per GPU-hour figure serves as a pricing reference point for users evaluating AI infrastructure costs.
The takeaway
The efficiency of large-scale inference is increasingly determined by software optimizations like speculative decoding and expert parallelism. Watch for future performance benchmarks released by competing labs to see if they can match DeepSeek's claimed daily token throughput.
Further reading
Explore the latest developments in large-scale model optimization in Artificial Intelligence.
Source note: This article includes information reported by TokenPost.
Live Poll
Do you believe current AI service throughput levels signal actual long-term profitability for the industry?
