NVIDIA’s Vera Rubin platform is engineered for high-performance scaling by employing a full-stack, co-designed architecture that integrates advanced hardware components with optimized software to function as a unified AI supercomputer. At its core, the platform combines seven distinct chip technologies: the Vera CPU, Rubin GPU, NVLink 6 Switch, ConnectX-9 SuperNIC, BlueField-4 DPU, Spectrum-6 Ethernet switch, and the Groq 3 LPU (Language Processing Unit). These components are not merely brought together but are “co-designed” for extreme integration across compute, networking, and storage, which allows the system to operate as a single distributed accelerator. The flagship Vera Rubin NVL72 rack, for example, unifies 72 Rubin GPUs and 36 Vera CPUs using the high-speed NVLink 6 interconnect, enabling them to act as one giant GPU for complex AI workloads. This integrated approach is designed to overcome traditional bottlenecks in data movement and communication, leading to significant improvements in performance metrics like inference throughput per watt and cost per token compared to previous generations.
Technical mechanisms underpinning Vera Rubin’s scaling capabilities include its sixth-generation NVLink, which provides a high-speed GPU interconnect fabric with 3.6 terabytes per second (TB/s) of bandwidth per GPU and 260 TB/s of connectivity within a single performance domain, drastically reducing communication latency and accelerating training processes. The Rubin GPU itself features a new Transformer Engine with hardware-accelerated adaptive compression for enhanced performance in AI inference. The Vera CPU, specifically designed for agentic AI and reinforcement learning, offers 88 custom Olympus cores with 1.2 TB/s of LPDDR5X memory bandwidth, excelling in data- and memory-intensive tasks. Furthermore, the platform integrates BlueField-4 DPUs and ConnectX-9 SuperNICs for accelerated networking, storage, and security tasks, while Spectrum-6 Ethernet switches handle high-bandwidth data movement. The Groq 3 LPU is purpose-built for low-latency inference in agentic systems, allowing fleets of LPUs to function as a single large processor. This comprehensive hardware integration is complemented by a modular software stack, including DSX Max-Q for dynamic power provisioning and DSX Flex for power-grid connectivity, which further optimizes the entire AI factory infrastructure for maximum token throughput per watt and continuous, high-intensity workloads.
This high-performance scaling is critical for agentic AI and other demanding AI applications that involve multi-step problem-solving, massive long-context workflows, and real-time interaction. For such applications, efficient data management, including the handling of vector embeddings, is paramount. A vector database like Milvus can leverage Vera Rubin’s high-throughput and low-latency infrastructure to store, index, and retrieve vast quantities of vector data efficiently. The platform’s ability to unify compute, networking, and storage into rack-scale systems allows for rapid processing of similarity searches and other vector operations, which are fundamental to agentic AI systems that retrieve context or knowledge from large datasets. By minimizing communication overhead and maximizing data throughput, Vera Rubin ensures that the entire pipeline, from data ingestion to model inference and agentic decision-making, operates at an optimized scale, thereby facilitating the development and deployment of complex AI solutions.