Beware of AI costs hidden in plain sight

Rapid and widespread AI adoption has most CIOs blind to what AI is really costing their organizations.

Nearly two-thirds of companies say employees have used AI without proper oversight, and almost half of large enterprises don’t have full insight into what AI tools employees are using, according to Protiviti’s 2026 AI Pulse Survey. Meanwhile, 77% of technology leaders say AI adoption is already outpacing their governance capabilities, per IBM’s 2026 Tech Leader Study.

“Combine the breakneck pace at which companies are looking to embrace AI with the low technical barrier to entry for using AI, and you’ve got an incredibly challenging space to keep tabs on,” says Andrew Retrum, managing director and global technology risk and resilience practice lead at Protiviti.

This isn’t shadow IT in the old sense, as financial exposure isn’t rogue employees signing up for ChatGPT. It’s the AI costs mounting in vendor renewals, usage-based consumption, and business unit budgets. A few CIOs have full visibility, but only because they built it into their architecture from day one. Others are still catching up. And some are finding that cost isn’t even the most important thing they can’t see.

Where the money hides

AI costs are showing up in three places that most organizations aren’t watching closely enough.

The first is vendor-embedded AI. Software providers are quietly adding AI features to existing tools, and the costs show up as renewal increases — not new line items. Some solutions are showing a 30% cost uplift as vendors embed AI functionality without upfront disclosure, according to Gartner research from September 2025.

The second is usage-based pricing. “The bulk of GenAI cost isn’t in the build, it’s in the run: inference, API calls, fine-tuning, and usage-based consumption that scales fast and unpredictably,” Gartner notes.

Phil Leslie, chief technology and innovation officer at Cornerstone Research, has seen this firsthand. “With Gemini, monitoring cost is largely irrelevant; the fee is fixed,” he says. “With Claude Code, costs are usage-based, and as adoption grows, so does spend. We noticed costs rising and have been building our dashboards accordingly.”

Getting a complete picture isn’t easy. “Getting a truly holistic view across Claude Code, Claude.ai, and our Office plugins is not trivial,” Leslie says.

The third bucket is business unit–led adoption. Various functions are buying AI solutions via credit cards or departmental budgets, outside IT’s line of sight.

Visibility by design

At Cox Business, Head of AI Eric Pace says the company has achieved full visibility into AI spend. But it required intentional architecture and governance from the start.

“We have 100% visibility into all AI spend, including SaaS-based modules through activations in BAU [business-as-usual] operations, new purchases and solutions, and all token consumption across the enterprise,” Pace says.

The key was centralization paired with clear accountability. “We were really intentional about building a model in our organization where AI is not a handoff, with one team building and another team simply receiving it,” says Pace. “We chose to centralize the AI function early to align with company-wide objectives, while business teams provide the workflow context that makes adoption real.”

Cox Business also built visibility into its architecture by default. “All AI traffic is routed through our AI gateway and runtime security solutions,” Pace says. “This includes all on-prem and cloud-based capabilities.”

That architecture extends to network monitoring. “We can see all traffic ingress and egress on the network and can also see what is running on our company devices,” he adds. “When we identify traffic patterns outside of our desired path or standard, we work with our people to find their way into compliance.”

Employees have access to the tools they need without creating ungoverned sprawl. “We have provided flexible ecosystem and capability sets that allow our people to operate with ‘freedom in a framework,’” says Pace. “And they generally find that they have access to everything they need.”

When cost takes a back seat

Not every organization is racing to solve the cost visibility problem. For some, other concerns take priority.

“Cost visibility is deliberately secondary right now,” says Leslie of Cornerstone Research. “The harder problem is the nature of our work.”

Cornerstone operates in high-stakes litigation, where expert reports must be error-free. “Figuring out how to unlock the benefits without compromising that trust is the primary challenge,” Leslie says. “Cost matters. It is just not the binding constraint in this phase.”

Leslie also points out a nuance that complicates cost optimization: The highest spenders are often the highest performers.

“Something like 80% of our costs come from 10% of our users — and that 10% tends to be our most experienced people, using AI for legitimate, high-stakes reasons,” he says. “You cannot just set a uniform ceiling without risking exactly the use cases you want to encourage.”

For now, Cornerstone uses caps that create friction when costs get high, paired with override mechanisms. “A high-cost user is often a signal of high-value work, not waste,” Leslie says.

Scale changes the game

The visibility challenge looks different depending on organization size. Leslie spent a decade at Amazon before joining Cornerstone and sees a clear contrast. “Amazon is not just bigger — it is more diverse,” he says. “At that scale, simple rules are often inefficient, but you reach for them anyway because managing complexity requires blunt instruments.”

At Cornerstone, he can be more surgical. “The feedback loops are short enough that I can have a direct conversation with a practice lead and understand in a few minutes what a team is trying to accomplish,” Leslie says. “That means we can tailor cost management to the specific situation — confidently incurring higher costs where we know it makes sense, rather than guessing.”

At Cox Business, scale required a different approach. “We centralized our capital investments and distributed enablement where teams are building enterprise applications outside of the center of excellence,” Pace says. Token consumption budgets are centralized but communicated constantly to larger consuming departments, and top consumers are interviewed frequently to understand value.

What’s working

For organizations still building visibility, it’s helpful to begin with the basics. “Start with an inventory — you can’t defend what you can’t see,” says Protiviti’s Retrum. “Assign clear ownership across IT, security, legal, and the business. And treat it as an ongoing discipline, not a one-time effort.”

Prioritization matters, too. “Don’t let perfect be the enemy of good,” Retrum advises. “Prioritize your highest-risk use cases first — where AI is touching sensitive data, customer-facing decisions, or regulated processes. Build your guardrails around those, then expand outward.”

Gartner recommends tagging AI purchases in procurement, tracking AI spend separately in IT financial management systems, and negotiating AI-specific cost clauses into cloud and SaaS renewals before the next renewal cycle.

At Cox Business, governance isn’t just about control — it’s about focus. “We have used ‘no’ liberally to keep our people focused on the things that will get us to value quicker,” Pace says.

Beyond the budget

For some organizations, the bigger risk isn’t runaway costs: It’s what happens when AI goes wrong.

“Shadow IT is not primarily a cost control issue — it is a reputational risk issue,” Leslie notes. “Depending on your firm and how AI is used, rogue use can cause real harm. That is the risk worth managing.”

Cost visibility matters. But for some CIOs, it may not be the most important thing they’re missing.