5 critical questions that define AI factory economics

As data centers evolve into AI factories, compute has shifted from a cost center to a revenue driver.

“Compute is revenue,” said Jensen Huang, co-founder and CEO of NVIDIA. “Without compute, there is no way to generate tokens. Without tokens, there’s no way to generate revenue. So, in this new world of AI, compute equals revenue.”

This reframe changes an organizations’ calculus. If compute is revenue, what do you optimize for? Here are 5 questions to consider:

1. Are you measuring what actually drives AI factory revenue?

Most AI factories are power-constrained, so tokens per watt dictate how much revenue you can generate and the cost per token impacts the AI factory profit margin.

But neither of these metrics should be evaluated at a single operating point. Batch jobs, real-time chat, and agentic workloads demand different points on the throughput-latency curve. AI chips that perform well at only a few points will underserve the full range of workloads.

Additional key operational metrics like time to first token (TTFT), mean time between interruptions (MTBI), and platform useful life are the bedrock of AI factory efficiency. They dictate how quickly an AI factory comes online to generate tokens, the reliability of its revenue streams, and its long-term ability to remain productive as AI workloads evolve.

2. How does agentic AI change what your CPU needs to deliver?

Data center CPUs have historically been optimized for parallel throughput, where more cores improve aggregate capacity. 

Agentic workloads run in loops and make different demands. The model reasons on the GPU, the CPU executes tool calls such as code compilation and data retrieval, and the result returns to the GPU so the model can reason again. Every step runs in sequence, gated by the one before it.

Per-core performance and memory latency determine how fast each step completes, which impacts the quality of service agents deliver and how well the AI factory stays utilized.

3. Is your networking and storage built for AI’s traffic patterns and data volumes?

Peak compute performance means nothing if the network cannot keep every accelerator productive. Networking requirements in an AI factory span three layers with performance demands that off-the-shelf Ethernet cannot deliver.

• Scale-up networking connects multiple accelerators and their memory with high bandwidth and low latency critical for today’s mixture-of-experts models.
• Scale-out networking enables high-speed direct data transfers between GPU memory across tens-of-thousands of servers, sustaining consistently low latency with zero jitter.
• Scale-across networking federates sites into a unified factory as power constraints push capacity across multiple locations, with intelligence and orchestration built in.

Storage must deliver more than capacity and throughput. Agentic workloads require fast, intelligent access to inference state and working memory across long context and multiple sessions. When storage paths can’t keep pace, GPU utilization drops.

4. Does your software stack hold up at scale and improve AI factory economics?

Turning hardware potential into realized performance requires a robust, proven software stack that optimizes every layer from compute primitives to inference frameworks to orchestration.

Open source software gives teams the flexibility to build, customize, and extend on a foundation shaped by a broad developer ecosystem. Strong enterprise-grade software captures that innovation while preserving the reliability for production AI. Moreover, software that delivers continuous performance gains at production scale reduces cost per token and extends the useful life of AI infrastructure.

5. Is security built into your AI data path?

A security breach can compromise customer data, model IP, or the integrity of agent decisions, and also result in downtime and lost token output.

Security must operate inline at AI factory speeds, across data at rest, in transit and in use. Storage must inspect agent behavior, enforce file and network access policies, and protect context memory in real time. At the compute layer, confidential computing with hardware-rooted attestation verifies workload integrity and protects models and data during inference.

How these questions shape extreme co-design at NVIDIA

These considerations from NVIDIA customers have shaped how we build. NVIDIA’s extreme co-design vertically integrates compute, networking, storage, and software to deliver the best performance, efficiency, resilience, and security to optimize AI factory economics. The NVIDIA platform is also horizontally open, ranging from NVIDIA MGX and DSX reference architectures to NVLink Fusion support for third-party XPUs to a broad open source software ecosystem.

The proof is in the performance leadership:

• NVIDIA Vera Rubin NVL72 delivers 10x more tokens per megawatt than NVIDIA GB200 NVL72.
• Vera Rubin’s cableless rack-scale architecture reduces tray assembly from 2 hours to 5 minutes with a 95% first-pass success rate, accelerating bring up and time to first inference. Combined with resiliency software, it sustains uptime. 
• NVIDIA Groq 3 LPX delivers up to 35x higher throughput per megawatt for ultra-low latency inference
• NVIDIA Vera CPU delivers up to 2x higher single-threaded core performance, 3x higher core-to-core bandwidth, and 40% lower memory latency to accelerate agentic AI.
• NVIDIA NVLink, now in its sixth generation, is purpose built for scale-up networking and delivers 3x lower latency and 10x higher packet rate versus off-the-shelf Ethernet.
• NVIDIA Spectrum-X Ethernet delivers up to 1.6x higher performance than off-the-shelf Ethernet and sustains up to 95% efficiency across deployments exceeding 100,000 GPUs.
• NVIDIA BlueField-4 DPU delivers 800Gb/s connectivity and 6x the compute of its predecessor, while the Vera BlueField-4 STX Storage Processor powers NVIDIA CMX to deliver up to 5x higher tokens per second for agentic inference.
• NVIDIA DOCA on BlueField-4 delivers in-silicon security, with runtime threat detection up to 1,000x faster than existing agentless solutions and file and network access policy enforcement.
• NVIDIA Confidential Computing secures models and data in use across every GPU and CPU in the Vera Rubin NVL72.
• NVIDIA software stack powers the world’s largest AI factories. On Blackwell, continuous optimizations reduced token costs for DeepSeek V4 by up to 5x within 1 month.

---