Is your architecture preventing you from calculating AI value?

To get ahead of claims of bias, I recommend managed services regularly. They get you to a working agent much faster, and they take scaling, patching, availability and capacity off your plate, none of which differentiates anybody. If you come to me with three engineers and an AI roadmap, my recommendation is to use serverless and managed services as much as you can. Running your own infrastructure requires a platform team and the maturity to go with it, which makes this a question of size and stage.

Those advantages are settled science. The other trade-offs materialize later, and while CIOs have weighed most of them before, they’re worth a brief mention.

Model selection is where this shows up in dollars. Open-weight models often run on infrastructure you control, while managed catalogs can lag availability or limit model choice. That matters when a newer model materially changes inference economics or performance. If you’re using a managed service that doesn’t offer the model you want, you can’t route to it through that service.

Stripe agreed in August to buy OpenRouter for more than $7 billion, a strong indicator of what the market thinks routing is worth. Of course, you can still route between models on a managed service, but you have to build the router into your own application code and run it yourself, outside the service you bought so you wouldn’t have to do that stuff.

The provider’s schedule also determines when you get access to new capabilities. For example, plenty of teams on their own infrastructure started building against MCP within days of its release, while stateful MCP server support didn’t reach AWS’s agent runtime until this past March, more than a year later.

It’s also worth noting that agents handing off to each other can introduce another execution or initialization cost, whereas on your own nodes you can keep resources warm. Enterprise authentication is not always one of the methods on offer, so you may have to build that path yourself anyway. And in regulated industries, proving where data can be a dealbreaker.

None of this is news to anyone who has run a platform, and none of it is disqualifying. You can put a number on each and decide it’s worth paying. The next one doesn’t work that way.

Let’s start with exploring what exactly your managed service provider tells you about your AI spend. The bill arrives on the provider’s schedule and can give you a detailed view of what you spent on a service, often by account or API key. That tells you where spend went up or down. It doesn’t necessarily tell you which workflow did it, which customer or which employee triggered it (not just created it). It also doesn’t enable you to map that spend to outcomes, productivity or revenue. A bill is not a measurement of value.

Going further, an AI agent isn’t a single service either. It’s the model call plus MCP servers, vector databases, APIs, prompt management and whatever else the workflow leans on, and that supporting cast can represent a substantial share of the cost. Without the ability to review those costs in isolation, you’re evaluating on estimates. You can’t say what a feature costs to serve, which accounts are profitable at the terms you signed, or whether the expensive part is the model or everything around it. So, you ballpark, and everything downstream inherits the inevitable errors.

The standard workaround has been to instrument the application, tracking each model call and carrying cost data through the workflow. I’ve built that many times and, on infrastructure you control, it works. But inside a managed runtime, you’re instrumenting someone else’s execution model. You can track the calls you make, but you may not be able to see the initialization you’re paying for, the orchestration between steps or the retries the platform runs on your behalf. You get detailed numbers for your own code, but less visibility into everything happening around it, and that’s often where the surprises are.

The expensive AI failures are also episodic rather than steady or predictable. An agent might loop on a retrieval it can’t satisfy, a workflow could revert to an expensive frontier model or a prompt change adds context and cost to every downstream call. If you’re reviewing the bill on a monthly cadence, you see that the AI number on the bill is bigger than last month, but don’t know why.

With real-time, per-request granularity you can instantly identify the workflow that’s misbehaving and it’s usually a quick fix. It also sets how soon you can intervene.

The way to get that granular, real-time view is to watch the kernel, using the same eBPF technology that security and observability tools use to see system calls without touching the applications above them. In an implementation like this, it can map compute and network activity back to the relevant process and request, then join outbound model calls to provider cost data. The advantage is that the view doesn’t depend on developers remembering to instrument every call.

The catch is that it only works where you control the host machine, a narrower set of places than most people assume. On AWS it means EC2, and container workloads on EC2-backed ECS or EKS nodes. It doesn’t mean Fargate, where you don’t control the host kernel. The same principle holds across clouds: customer-controlled VMs and Kubernetes nodes can give you access to the kernel, while serverless and fully managed runtimes generally don’t.

It’s a firm line in the sand. You either have the option, or you don’t. There are no workarounds here. That alone doesn’t make managed services a mistake, and provider telemetry and application instrumentation still matter, they just can’t give you the same infrastructure-level view. If your inference runs where you don’t control the machine, you get as much infrastructure-level visibility as your provider chooses to share, and that ceiling remains capped even as the number of agents and the value of visibility grows.

Today, price per token is the standard unit for AI spend, but it’s the wrong one for agents. Agents burn far more tokens than chat; input drives most of the cost, and token use on the same task swings widely between runs, so cheaper tokens can still produce more expensive tasks. Cost per completed task is what tracks value.

But pricing a task means matching spend to its specific workflow and a measurable outcome. On infrastructure you control, you can get that visibility. On a managed runtime, you’re mostly inferring from an aggregated bill.

The last two years rewarded shipping. The next phase rewards knowing what each task costs and what it’s worth, and companies that can’t see the work will be competing against ones that can.