Where IT leaders find strength and opportunity in the age of AI

With vision comes perspective, and over a distinguished career, IT and digital transformation leader Niraj Bhatt has held may titles, and earned three consecutive CIO 100 awards since 2023.

As a storied advisor for startups and Fortune 500 companies, helping them navigate the unpredictability and fluidity of AI, Bhatt knows how emerging tech is rapidly reshaping the way organizations build products and deliver value, and how challenges shift as companies move from experimentation to real-world deployment.

AI, of course means a lot of different things to different people, and also for frictionless startups and large enterprises. For the former, speed is a huge asset, allowing them to punch above their weight. But it also means they need lightning fast reactions when landscapes shift. “The same speed can also hurt them when larger AI companies release new offerings that disrupt what startups are building,” he says, referencing recent moves by Anthropic and Google.

On the enterprise side, the conversation is more about scale and risk. Many large organizations have moved past the POC stage and now wrestle with the realities of putting AI into production.

Cost for both is naturally a recurring theme as organizations scale up AI efforts, and true expenses become clear only after the initial excitement fades. “Every input and output token, and the model you’re selecting, add up,” he says. Some customers like Open AI, he adds, get throttled because their usage, volumes, and costs are growing so fast, making planning, observability, and monitoring critical for any team moving beyond experimentation.

So understanding the full software development lifecycle is also vital. Therefore, before committing to production, he helps clients see the big picture, and make sure they understand technical requirements as well as operational and financial implications. “The cost picture isn’t just about usage, but scale and the model choices teams make,” he says.

Bhatt also discusses effective approaches to AI and enterprise IT, technology leadership, and the evolving role of today’s CIOs. Watch the full video below for more insights, and be sure to subscribe to the monthly Center Stage newsletter by clicking here.

On AI hype: If you can’t explain something to someone who’s eight or 80, you don’t really understand it. It’s gone from LLMs, to RAG, to agentic AI, and now the essence is all about tokens. It’s predicting that next token and understanding that is key. So when LLMs came out, they were good at doing that on the data on which they were trained. When the enterprises looked at it, they wanted to make those LLMs work for their data. And the question became how to provide our data and context. It’s about building the right context for the LLM. Agentic AI is similar and that’s where the RAG evolution came in, in that I’ve got my data because every LLM has limitations in terms of how much context it can carry.

There are ranges of LLMs, where Google has the highest in regard to the context window size and what they support. Agentic AI is more action oriented, though. LLMs rely on the metadata you provide for the tools. Then they’re doing token prediction in that whatever I’m looking for, I should use a specific tool. Then it’s the infrastructure underlying which LLM it relies on to invoke the agent. So if you try to explain the microservices to a person, you’re going to struggle. But it’s very important to understand the evolution and that’s where you can cut through the hype. Understanding in this context is key.

On navigating challenges around talent: What I’m seeing on the IT side is there’s so much cognitive load, so how do we empower people to build solutions with the right mix of products and platforms? I think it’s about democratizing AI for the entire organization. Your talent strategy is everyone, all inclusive, starting from interns, the business and tech sides, CEO, everybody.Like your customer success or revenue officers, you need a talent strategy because in the end, IT alone isn’t going to be in a position to deliver for everyone in the organization.

AI has the potential to make everyone in the organization more productive. You have to plan that and facilitate broad innovation across the organization.That’s where the talent strategy, and working with HR and the people officer becomes very important providing those tools. One part of it is training, but how do I build an agent for a receptionist receiving calls, for instance?I’m not going to rely on vibe coding or things of that nature. But what are the tools? Where do I go, where do I host this? I think through that entire ecosystem beyond copilots. That’s where innovation can kick in, and that broader talent strategy is something I’m working with my customers on.

On collaboration: I heard a panel discussion recently, and a question was asked about what’s the number-one trait CIO needs to be successful at in the world of AI, and the answer was collaboration. You need to bring everybody together, move forward together, and make sure everybody’s on board. And in my mind, simplifying that is more like systems thinking when you operate, just bringing everybody along and ensuring they’re meeting outcomes.

But maybe what’s more important is managing expectations. Because if you’re a CIO, there’s a tremendous amount of pressure to deliver and have a rock solid AI strategy. So what I’m doing with my customers is get the board, CEO, and CFO into a room and help them understand what I’m talking about, the evolution, and what’s the art of possible. You don’t want to be a CIO who thinks I have a hammer and everything is a nail. Having buy in from the senior leaders is essential to know you’re headed in the right direction. You’re not reacting to pressure from top leadership, but driving and becoming the change agent for good for the company.

On navigating AI: It’s interesting times. I’m covering a spectrum of startups, non-technical and technical founders, and advising Fortune 500 companies. What I’m seeing is they love the velocity and momentum because that’s what they’ve always wanted, and AI is providing that. They’re able to bring their products to markets very quickly, so something that would’ve taken three years a couple of years ago is probably now taking them three months. There’s a lot of excitement there. But on the flip side, the same velocity is also hurting them. There are so many frontier AI companies getting disrupted. OpenAI, for instance, has offerings in sales and marketing, and Google has an interactive video model. So a lot of startups working in the marketing space are getting stuck. A lot of what I’m focused on is working with founders, helping them pivot in the gen AI space, ensuring their systems and products are built and structured in the right manner.

And on the enterprise space, what I’m seeing is the POC wave, and people have seen the value. There’s some excitement but now the struggle is getting them to production. That’s where you run into cost, latency, legal compliance, privacy issues, and customer concerns that if we get tickets to production, how’s it going to look and how are we going to scale. So engineering and product teams have to be ably supported by the enterprise architecture and R&D teams. I then help them get up to speed and build that internal platform product for the production workloads. It’s exciting times on both sides.