The token debate: What CIOs can learn from the laws of thermodynamics

What if the next breakthrough in Enterprise AI doesn’t come from computer science alone?

What if it comes from applying principles that physicists have understood for more than a century?

According to Gartner, rising token-driven AI spend is straining budgets and challenging cost justification. As organizations race to deploy generative AI and agentic systems, token consumption dominates nearly every executive discussion: How many tokens did we use? How much did inference cost? Can we reduce our AI bill?

These are important operational questions. But they are not the strategic questions.

I believe the economics of enterprise AI can be viewed through the lens of three well-established principles from thermodynamics: the conservation of energy, entropy, and exergy.

While these principles describe physical systems — not AI —they offer a useful way to think about how organizations should measure AI success.

Principle 1: Value is created through transformation

The 1^st Law of Thermodynamics tells us that energy cannot be created or destroyed. It can only be transformed.

Enterprise AI presents a similar management lesson: Tokens are not valuable because they are consumed; they become valuable only when they are transformed into business outcomes: A faster loan application decision. A better customer experience. Faster and more accurate software. Reduced fraud. Higher employee productivity. A new product. A strategic insight.

The executive question therefore is not, “How many tokens did we consume?” It is: “How much business value did those tokens create?”

This leads to a new executive metric: return on tokens (ROT).

Just as organizations measure return on investment, they should begin measuring the business value generated for every million AI tokens consumed.

The organizations that win will not necessarily consume fewer tokens. They will generate more value from every token they use.

Principle 2: Every transformation creates waste

The 2nd Law of Thermodynamics teaches us that every energy transformation introduces inefficiencies.

Some energy inevitably becomes less useful for doing work.

The same pattern appears in enterprise AI: Not every token contributes equally to business outcomes.

Some are spent on:

• Repeated prompts
• Oversized context windows
• Redundant reasoning
• Hallucinations requiring correction
• Multiple agents performing the same work
• Expensive models solving simple problems

Those tokens are not “lost.” They simply produce very little business value.

I think of this as token entropy. Every enterprise deploying AI will experience it. The goal is not to eliminate token entropy completely — that would be unrealistic. The goal is to continuously identify it, measure it and reduce it. Because every unnecessary token represents an opportunity to improve both cost and business performance.

Principle 3: Useful work matters more than energy consumed

Thermodynamics introduces another important idea: Exergy.

Unlike energy, exergy measures how much energy can actually be converted into useful work. Two systems may consume the same amount of energy while producing dramatically different results.

The same is true for enterprise AI.

Imagine two companies each consuming one billion tokens. One produces meeting summaries. The other transforms claims operations, accelerates software delivery, detects fraud, improves customer retention, and creates new revenue opportunities. Both consumed the same number of tokens. Only one extracted significantly more business value.

Borrowing this concept as a management analogy, I call this token exergy.

Token exergy represents an organization’s ability to convert AI intelligence into meaningful business outcomes:

• High token exergy means AI is solving important business problems.
• Low token exergy means AI is generating activity without creating proportional enterprise value.

The distinction matters, because activity is not the same as impact.

A new responsibility for CIOs

For years, CIOs have monitored infrastructure: Cloud costs, storage, network utilization, GPU consumption.

These metrics remain important, but they tell only part of the story.

Token usage needs to be measured, planned, optimized and governed with the same discipline as any other cloud resource. This means that the next generation of CIO dashboards should answer different questions:

• What is our return on tokens?
• Where is token entropy reducing our effectiveness?
• How much token exergy are we generating?
• Which AI initiatives produce the greatest business value?
• Which use cases create the strongest competitive advantage?

These are no longer technology metrics. They are business metrics.

The next generation of CIOs will not simply deploy AI. They will manage an economy of intelligence.

Their role will resemble that of a portfolio manager — allocating AI capacity where it creates the greatest enterprise value, reducing waste and continuously improving the productivity of every autonomous workflow.

That responsibility cannot be fulfilled by dashboards alone.

It requires an intelligent layer capable of observing, learning and optimizing the entire AI  ecosystem. Three-layer enterprise agentic architecture Will enable this.

The next competitive advantage

Every major technology revolution eventually shifts from measuring inputs to measuring outcomes:

• Factories stopped measuring coal consumption and began measuring productivity.
• Cloud computing evolved beyond server utilization to business agility.
• Digital businesses measured customer acquisition costs and lifetime value.

Enterprise AI is approaching the same inflection point. Organizations that focus only on token costs will optimize for efficiency. Organizations that measure return on tokens, minimize token entropy and maximize token exergy will optimize for business transformation.

That is a fundamentally different objective. And I believe it will separate AI leaders from AI followers.

Because in the end, the future of enterprise AI will not be determined by how many tokens an organization consumes. It will be determined by how effectively those tokens are transformed into lasting business value. The AI adoption spending spree is over. Time to focus on value.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?