AI data centers are becoming fortresses — and that’s the point

10 years ago, in NVIDIA’s developer conference, The GPU Technology Conference (GTC), NVIDIA CEO Jensen Huang had already declared that, “In this era, software writes itself and machines learn. Soon, hundreds of billions of devices will be infused with intelligence. AI will revolutionize every industry.” So AI is being viewed as the engine of intelligence in the 21^st century and that engine requires a new kind of factory — a modernized AI data center.

Today, we have witnessed the AI data center’s modernization journey, which is a dual-track race to achieve unprecedented computing scale and speed while erecting an impregnable cybersecurity defense. This fusion of acceleration and cybersecurity is creating a new paradigm for AI data centers’ modernization.

 In 2024, U.S. Secretary of Homeland Security Alejandro Mayorkas said, “The choices organizations and individuals involved in creating AI make today will determine the impact this technology will have in our critical infrastructure tomorrow.” The cyber threat landscape is indeed increasingly complex and evolving. Our critical infrastructure, including the data centers powering our AI future, is a target of choice for adversaries. The modern AI data center must be both the most powerful engine and the most formidable fortress.

This article will take a holistic past, current and future view on how the critical role cybersecurity would play in the AI data center modernization.

Part 1: Historical foundation  — Pre-AI data center’s security model (1990s – 2010s)

Pre-AI data centers operated on a historically perimeter-defined defense security model, just like the castle-and-moat paradigm. Security was largely perimeter-based and static, with the following typical characteristics: 

• Hardened perimeters. Cyber defense controls implemented on the data centers at that time focused on firewalls, intrusion detection systems (IDS), virtual private networks (VPNs), etc at the network edge. The internal corporate network was often considered trusted, while anyone sitting outside, crossing the perimeter, would be considered untrusted by default.  
• Data-at-rest encryption focus. Encryption was primarily implemented for sensitive data stored in the databases or on the physical tapes. Data in motion, especially East-West internal network traffic, between servers inside the data center, was often unencrypted, based on the assumption of a secure internal network. This might allow lateral movement of bad actors after they penetrate the corporate network.
• Manual compliance and audits. Cybersecurity check was usually a checklist-driven manual approach at the last step before going live in production, with an eyeball check and a tick involved. This has created an inconsistency in compliance and audit and slowed down the agile development of the modern digital transformation.

Part 2: Evolving change — ZTNA security model (2010s – present)

The shift to accelerated computing in the data center did not just change the performance; it exploded the attack surface and demanded a parallel revolution in cybersecurity, together with the data center modernization. So the new security model, zero trust network architecture (ZTNA), that follows the principle of “never trust, always verify,” has received industry recognition. Instead of relying on traditional network perimeter security, ZTNA mandates that all access requests, regardless of their origin, are strictly validated before granting permission. So this wave of security framework shift is aligning closely with the shift of AI data center modernization, for example: 

• The AI accelerator is the new threat vector. GPUs, TPUs and other AI accelerators are complex systems-on-a-chip (SoCs) with their own firmware, drivers and memory spaces. All these are high-value targets in front of cyber attackers. For example, a compromised GPU firmware could poison model training at scale, leak proprietary model weights, or create a persistent backdoor for bad guys. So today, cybersecurity has to extend to the silicon level, require a hardware root-of-trust and secure boot for every AI accelerator. 
• The software supply chain is under target. Modern AI development relies on the interconnected software supply chains, including AI frameworks (PyTorch, TensorFlow), libraries, containers and pre-trained models, etc. This necessitated more rigorous software composition analysis (SCA), artifact signing and a vetted container registry for AI workloads. The SolarWinds supply chain attack, although not AI-specific, was a stark lesson for the whole industry. 
• From data lake to data ocean for AI training data. AI training data is often massive, unstructured and aggregated from countless sources. The terminology we used years ago, data lake, is not big enough to host them now, as we need a data ocean. AI training data is a prime vehicle for poisoning attacks, where maliciously crafted training samples can bias or even break a model. Data security for AI training data has evolved from simple access control to include data lineage tracking, integrity checksums, etc to detect and purge toxic data before it can corrupt an expensive training model. 

Part 3: Status quo — The fusion of GenAI and security (Nov 2022 – present)

On Nov 30, 2022, OpenAI launched ChatGPT to the public, which reached 100 million users just within two months. The subsequent generative AI boom has turned AI data centers into crown jewel assets, attracting both state-sponsored and criminal actors.

As cybersecurity expert Bruce Schneier quoted, “Security is a process, not a product.” For AI data centers, this process is now continuously integrated and measured at the same speed as the AI workload itself. So, what is the current fusion of GenAI and security? 

• An AI model is the new intellectual property (IP) to be secured. The primary asset for an AI data center is no longer just the data itself; it’s the AI model being trained and has been trained. Theft or leakage of the expensive AI model is a top-tier threat to be concerned about. That’s why modern AI data centers have implemented robust security governance like strict access controls, logging of model interactions, implementing techniques like watermarking to trace leaked models, etc. They are the important guardrails to prevent AI-specific cyber attacks like prompt injection, data leakage and malicious use. 
• Identity becomes the new perimeter to defend. With ZTNA further evolving in the AI data enterprise modernization, AI-related non-human identities will be increasing. Every AI service and AI workload needs a cryptographically verifiable and strong identity.
• Using AI to fight AI. Security operations centers (SOCs), the heart of modern security defence, are integrating AI to defend AI. For AI data centers, an example could be the pattern of a GPU suddenly accessing memory that is consistent with a weight extraction attack, or an AI model training pattern that deviates from its expected standard data access pattern; all these would remind cyber professionals that bad guys are attempting to poison our AI service.

Part 4: Future landscape — The AI-augmented and cyber-resilient future (present – 2030s)

The future of AI data centers’ security would be augmented with AI, intelligent and resilient by design. The following are a few predictions for the next decade: 

• AI-on-AI defense. The next generation of security will be autonomous and intelligent defensive AI systems running inside the AI data centers. These defensive AI systems will adaptively configure firewall policies, predict attack vectors dynamically by real-time analyzing global threat intelligence and local configuration and automatically initiate incident response inside the AI data centers, like quarantining compromised GPU(s).  
• Confidential computing becomes the new standard. Encryption of data in use is still needed, but will move from a niche cyber practise for sensitive workloads to a default expectation for all AI training and AI inference. Confidential computing would protect data in use by isolating it within a hardware-based trusted execution environment (TEE) in CPU/GPU, encrypting data in memory so even cloud providers or administrators can’t access it, and solving the security gap left by data-at-rest and data-in-transit encryption in the past. Key technologies from chip giants like Intel SGX and AMD SEV create these secure enclaves, allowing sensitive computations for AI to run securely in the cloud or at the edge. 
• IT and OT (operational technology) security convergence. The increasing power and cooling demands of AI Data Centers make the physical facility of data centers a new cyber target. For example, an attack on the building management system of an AI data center could overheat and destroy multi-million dollar AI racks in the data centers. So, the more converged security orchestration could integrate IT and OT security with automation and intelligence would be needed.

Security is the fortified engine of intelligence for AI data center modernization

The AI data center modernization journey is a tale of two interdependent evolutions — increasing computing power and energy efficiency demand versus increasing sophisticated cybersecurity. One cannot succeed without the other.

AI data center modernization needs an ecosystem that must be simultaneously open enough to foster the creativity of AI development and closed enough to defend against smart cyber attackers with the help of AI.  Such adaptive defense needs the evolving cyber security be embedded into the AI model, data, system and even silicon, as security is the fortified engine in the intelligence age of AI data center modernization.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?