NVIDIA Releases AI Cluster Runtime for Distributed Inference

The new runtime standardizes the deployment workflow for Kubernetes clusters, supporting H100 through GB300 architectures.

Updated on Oct. 30, 2026 in Artificial Intelligence

Isometric editorial illustration of stacked server modules in muted tones, representing the structure of a distributed computing system. — NVIDIA launched its AI Cluster Runtime v1.1.2 to standardize deployment workflows for distributed inference tasks across Kubernetes clusters, supporting modern GPU architectures. AI Illustration. Upload story photo >

Live Poll

Is managing enterprise-grade AI infrastructure becoming too complex for typical Kubernetes operators to handle?

NVIDIA has released the AI Cluster Runtime (AICR) v1.1.2 to streamline distributed inference deployments on Kubernetes clusters. The tool automates deployment workflows into snapshot, recipe, bundle, and validation phases, currently supporting NVIDIA H100 GPU environments.

Why it matters

Distributed inference tasks require complex synchronization of hardware and software configurations across large clusters. AICR addresses this by standardizing the deployment pipeline, ensuring compatibility with evolving hardware like GB200 and GB300 accelerators.

AICR v1.1.2 automates deployment through a four-phase workflow, requiring Kubernetes 1.34 or later to utilize Dynamic Resource Allocation. It currently maintains a catalog of optimized inference recipes for GB200 and GB300 hardware architectures.

The players

NVIDIA

A designer and manufacturer of graphics processing units and accelerators for data center AI and distributed computing.

Amazon Elastic Kubernetes Service

A managed Kubernetes service provided by AWS used for deploying, scaling, and managing containerized applications.

The details

The AI Cluster Runtime operates by resolving target configurations and rendering deployment files for integration with tools like Helm, a package manager for Kubernetes. The process utilizes the Gateway API Inference Extension to route requests through the Dynamo Endpoint Picker Plugin. The Dynamo Platform 1.4.2 architecture uses TCP for the request plane and ZMQ for key-value event synchronization, providing a structured approach to managing distributed inference nodes.

October 30, 2026: The AI Cluster Runtime release was documented.

The Tech Race

This release marks an effort to standardize inference operations for high-density GPU clusters. It aligns with the broader industry transition toward native Kubernetes support for accelerated computing hardware.

Engineers and infrastructure teams managing inference clusters on Amazon EKS can now implement the AICR v1.1.2 workflow to automate configuration management. This is designed for organizations utilizing H100, GB200, or GB300 accelerators within Kubernetes environments.

The takeaway

Developers should watch for further updates to the inference recipe catalog as support for newer hardware like the GB300 matures. Tracking the evolution of the Dynamo Platform Helm chart versions remains the primary method for maintaining cluster deployment stability.