Choosing your AI stack: The benefits of vendor lock-in

AI has emerged as a top priority for businesses and a vehicle for transformation, as evidenced by Accenture research: 97% of executives believe AI will transform their company and industry. But as companies move from AI pilots to scaling AI across the enterprise, we have had repeated conversations with CIOs and technology leaders who are arriving at the same uncomfortable realization: AI stack decisions are not easily reversible.

Unlike earlier eras of enterprise IT, where abstraction layers insulated applications from hardware choices, today’s AI stack—the infrastructure, technologies and frameworks that powers AI systems – tends  to be tightly co-engineered, with stronger dependencies in the underlying compute layers. Choices made about models, runtimes and compute platforms now shape cost structures, performance ceilings and strategic flexibility. AI-ready infrastructure has re-emerged as a new source of differentiation, and with it, a new kind of vendor lock-in.

At the center of this shift is the move from training – building AI models – to inference, where those models are used in production to generate outputs from new data. While early attention focused on the cost of training large models, enterprises are now scaling AI across the organization, running models continuously across workflows. This shift significantly changes the economics of AI.

For instance, agentic AI is reshaping infrastructure architecture and platforms because inference is becoming persistent, stateful and increasingly data intensive. As AI Factories scale, the focus is shifting from peak model performance toward sustainable token economics, where the key differentiators are lowest cost per generated token, power efficiency and infrastructure utilization at scale. In this environment, achieving those outcomes requires full-stack optimization across compute, networking, memory, storage and data fabrics, curated and integrated across ecosystem partners. Secure multitenancy and confidential computing are becoming core design principles, and enterprise AI is now ready to be industrialized at scale.

Modern AI infrastructure is a strategic bet

What makes AI infrastructure different is not just scale, but integration. Modern AI systems are built on tightly co-engineered stacks where GPU accelerators, high-bandwidth interconnects, compilers and runtimes are designed in tandem to maximize throughput and efficiency for AI workloads.

To get the massive computing power required for AI, providers design their hardware and software to work exclusively with one another. This has shifted enterprise decision-making from choosing hardware one piece at a time to committing to ecosystems. And that commitment carries consequences.

In traditional IT environments, applications could also generally move across environments with a manageable amount of effort. In AI systems, that assumption breaks down. What appears portable at the model or application layer often depends on deeply optimized components underneath that layer, such as memory handling and compiler frameworks like CUDA or ROCm that are fine-tuned to specific hardware.

We find it useful to think about AI systems as a layered structure:

While upper layers retain some flexibility, dependencies increase as you move downward. Changing your foundational AI provider often means having to rebuild and re-optimize large portions of your technology from scratch.

This is why infrastructure decisions in AI feel less like procurement choices and more like strategic, high-stakes bets.

Why switching AI platforms is harder than it looks

In theory, switching platforms should be straightforward. Models can be retrained, applications rewritten, and infrastructure replaced. In reality, the cost of switching extends far beyond hardware or licensing.

• The first challenge is engineering effort. Migrating to different platforms requires engineers to revalidate model behavior, re-tune inference pipelines, and rebuild performance baselines. During this period, teams spend most of their time stabilizing and not innovating.
• The second challenge is hidden dependency. Over time, system optimization becomes tied to a specific stack. This might include latency expectations, batching strategies, orchestration logic and even human workflows. These ties are not always obvious, but they shape how systems behave in production.
• The third challenge is timing. There is never a convenient time to migrate, especially factoring in rising AI infrastructure and inference costs, competitive pressure or scaling demands. Organizations are often forced to switch platforms precisely when disruption is hardest to absorb.

Rethinking performance vs control

Despite these barriers, organizations do switch. In our experience, this typically happens under three conditions.

One common trigger is when the opportunity cost of staying begins to outweigh the cost of leaving. As performance gaps widen across competing ecosystems, inefficiencies accumulate to the point that remaining on the current platform is no longer viable. Another driver comes from shifts in vendor dynamics. Pricing volatility, supply constraints, or misalignment in product roadmaps can introduce risks that force a re-evaluation. Finally, regulatory requirements, data sovereignty constraints or geopolitical shifts can force platform changes regardless of technical preference.

Across all three strategies, one principle stands out. Lock-in is not inherently negative, and openness is not inherently superior. Timing matters more than ideology.

Given these dynamics, the central question for CIOs is not how to avoid lock-in, but how to manage it deliberately. This represents a significant shift in strategies that previously considered vendor lock-in as a detriment. In practice, we see three broad approaches emerge, each reflecting a different balance between performance and control.

Some organizations take a performance-first approach. They optimize deeply within a specific ecosystem because performance directly drives business outcomes. Eli Lilly’s AI Factory is a strong example. The company has invested heavily in a tightly integrated NVIDIA-based stack to maximize throughput and utilization. In this case, infrastructure is a competitive lever and not merely a support function. Higher switching costs are accepted because near-term performance advantages are decisive.

Others lean toward a portability-first model. These organizations prioritize flexibility, governance, and long-term independence over absolute performance. BNP Paribas illustrates this well through its internal LLM platform built on open-source models and controlled infrastructure. By retaining ownership of the stack, the bank ensures data sovereignty, regulatory alignment and predictable cost.

A growing number are adopting a hybrid approach. Rather than applying a single strategy across the enterprise, they segment workloads based on sensitivity to performance, cost and governance. For example, in late 2024, JPMorganChase outlined its approach at a leading cloud and technology conference. It described combining a firm-wide internal AI platform with cloud-based services to move generative AI into production at scale. This reflects a broader enterprise pattern of pairing internally controlled environments with external ecosystems to balance control, scalability and cost.

A performance advantage is only valuable if it lasts long enough to justify the lock-in it creates. Similarly, portability only matters if the ecosystem evolves in ways that make switching worthwhile. This is where many organizations struggle. They evaluate platforms based on current benchmarks rather than the direction of the ecosystem.

In practice, we encourage leaders to track a set of evolving signals. These range from the maturity of open compiler ecosystems and improvements in cross-platform runtimes, to shifts in performance per watt and increasing regulatory focus on sovereign AI. Together, these indicators help determine whether the industry is moving toward convergence or further fragmentation.

Conclusion

AI is forcing a reset in how technology leaders think about IT architecture. The goal for CIOs is no longer to eliminate dependency, but to choose it consciously and manage and revisit that choice over time.

In our experience, the most effective organizations treat this as a dynamic problem. They evaluate where performance truly differentiates them, where flexibility protects them, and how quickly those boundaries are shifting. They also recognize that some degree of re-platforming is inevitable and plan for it, rather than treating it as a failure.

Ultimately, AI infrastructure strategy is not about optimizing for today’s conditions. It is about getting ready for where the ecosystem is going next. The leaders who navigate this well are not those who avoid lock-in entirely, but those who understand when to embrace it when to limit it and when to move beyond it before the market forces that decision on them.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?