AI agent sprawl pressures CIOs to recalibrate governance

Every Friday, Bret Greenstein, CAIO at consulting firm West Monroe, holds a company-wide meeting to share what’s happened in AI over the past week. He also spotlights one employee at the firm who’s created their own AI agent from the ground up, which lives in the company’s internal AI store. Since the store launched in May, more than 200 employees across departments — many without any technical, engineering, or coding background — have created over 550 agents.

“About 15% of our firm builds all the time now,” Greenstein says. “That’s a huge population.”

Enabling employees to spin out their own agents has become popular at many firms. Staff have built hundreds of agents at software company Blackline, for instance, and Microsoft has deployed more than 500,000 internal agents to help employees streamline workflows. Gartner also anticipates that by 2028, global average Fortune 500 companies will have more than 150,000 agents.

Employees know the intricacies of their work, the biggest pain points, and time drainers, so they can build solutions that address those specific issues, according to Greenstein. It also creates enthusiasm, empowers employees, and fosters innovation among the workforce as they build from the ground up.

That said, there’s been a pivot over the last six months, says Michael Murphy, partner and AI practice lead at global management consulting firm Adaptovate. When agentic AI first came on the scene, companies went all in, pushing to build and agentify nearly anything they could. In recent months, however, the narrative has shifted to getting a handle on agent sprawl, assessing the value agents deliver, and keeping costs in check.

“We’re really at this interesting inflection point where clients are having to figure out if we built the right agents, and are they delivering the value we expected,” Murphy says.

Today, tech leaders face a three-way squeeze, says Tiago Azevedo, CIO at AI-powered low-code development platform OutSystems. From the workforce side, many employees ask for permission to use more AI, but the CFO says token usage is becoming too big an expense on the balance sheet, and the CEO wants to see innovation and results from workforces using AI agents.

“I think that’s the biggest challenge for a CIO,” Azevedo says. “Let people take advantage of the technology but in a way that’s cost-effective and actually brings ROI.”

Building in a controlled environment

Employees have built myriad tools to aid their daily workflows. Azevedo’s company launched an agent dubbed Signal Sam, which searches databases of prospective customers, and gives account executives information to pitch them. Murphy and Greenstein also mention finance departments using agents to scan and categorize invoices, HR conducting a first pass on résumé screenings via agents, legal teams utilizing a self-service agent for NDAs, and marketing employees building agents that pull and analyze data from CRMs. These tools are often created by non-technical employees who’ve never written a line of code.

With so many agents popping up, CIOs need a way to oversee them, and ensure they meet corporate standards but without choking innovation, Azevedo says.

He recommends role-based access controls embedded into tools and configured behind the scenes. “So we allow them to use, but in a way that’s governed and controlled, because that’s our duty to the organization,” he says.

Ivan Burazin, CEO and co-founder of open-source developer platform Daytona, advises CIOs to treat agents like employees. “You’re not going to bump into them in your local Starbucks,” he says, “but you give them tasks and they have access.”

So set up agents with specific credentials, like how an organization would grant access to a new hire, with a laptop locked down with organization security protocols, Burazin adds. He also recommends sandboxing, in which agents operate in isolated machines with scoped credentials and firewalls so the sandbox prevents agents from accessing corporate systems or data outside allowed perimeters.

Organizations could use an internal ticketing system as well where employees wanting to build agents request a new identity for them, Burazin says. That way, tech leaders maintain visibility and governance over new agents.

“If something goes haywire in audit logs tomorrow, you can see it’s that agent versus an actual human,” Burazin continues.

He acknowledges that giving employees what feels like free rein to build and run agents can induce stress for CIOs and CISOs. But if a company doesn’t proactively establish tools, employees are apt to privately build AI in the shadows. As long as agent development happens within established confines, it won’t create problems organization wide.

“If you just enforce the security posture that you would for humans, you’ll save yourself a lot of headaches,” Burazin says.

When creating the AI store, Greenstein started by certifying tools for chat, code, data analysis, and other tasks, and then trained employees and made the tools broadly available to use. That process created guardrails and an inherently secure building environment. It also allows tech leaders to continue to monitor prompts and activity.

Now, tech teams review what’s been built in the AI store and flag any agents that excel. If employees have built 10 project management tools, for example, the leader will tag what they deem the best one. That gives employees the option to use existing agents or build a separate version for themselves.

More agents, more tokens

Over the last three to six months, Azevedo has been hearing from customers that their biggest hurdle is agent sprawl and the increasing cost those agents bear due to token usage.

In mid-July, OpenAI published a guide around useful work per dollar, sharing how leaders can look at tasks completed, time saved, and decisions improved to determine if their AI investments are bearing fruit. In addition to using the guide, Murphy suggests comparing the labor time and cost to conduct a manual task against time saved by using an agent, including which type of model the agent requires.

A cheap flash model, for instance, could be easy to justify the cost. “If it’s a very expensive Opus or Fable level model, that’s going to be a lot more challenging of a cost equation,” Murphy says. He adds that making this comparison isn’t about replacing the workforce but swapping “knucklehead admin work” for more engaging, human-centric work. This change may also require some organizational restructuring, such as CIOs and HR leaders working more collaboratively to handle change management as job responsibilities shift. Without the workforce optimized to work with agents, organizations won’t see the promised ROI of use cases, Murphy says.

West Monroe also informs its employees on the costs of different models. Without knowledge about tokens and costs, many employees defaulted to the highest-end model for any tasks before understanding that models come with different price tags. “We started educating people on the various relative costs of different models, and they immediately adjusted behavior, and our cost dropped,” Greenstein says.

While strictly quantitative returns are one way to measure ROI, Greenstein also thinks about return in a qualitative sense. “What does speed get me?” he asks. If someone in the firm is able to follow up with a client in hours because of an agent’s assistance, rather than days or weeks without one, the client will be impressed, and the firm might win their business over a competitor.

“Tokens will cost money no matter what,” he says. “But if you maximize the return, it’ll far outweigh the cost.”