Don’t let AI negotiate with reality

We are all transforming now.

Some companies have formally named transformation programs. Others are being transformed by a new regulation, an AI mandate, a cyber event, a weather disruption, a change in customer behavior, a competitor’s move or an urgent demand to reduce costs.

The label is almost beside the point. The operating assumptions keep changing, and the company has to change with them.

Accenture’s Change Reinvented research found that 95% of organizations had undergone at least two transformations in three years, while only 30% of C-suite leaders expressed confidence in their organizations’ change capabilities.

More recently, a McKinsey Global Survey of more than 1,200 executives and managers found that 40% expect their current business models to require significant change within three years simply to remain economically viable.

Transformation is no longer an event that temporarily interrupts normal operations. It is becoming normal operations.

That changes the role AI is beginning to play. We are not using it only to draft emails, summarize documents or write code. We are increasingly asking it to interpret complex situations, identify options, recommend priorities and influence consequential business decisions.

I believe that can be enormously valuable. I also believe it requires a boundary we have not defined clearly enough.

AI should help us understand reality. It should not be allowed to negotiate with it.

The impossible request

I have sat in versions of this meeting many times.

The company must reduce spending by 10%. A regulatory deadline cannot move. The CEO has declared AI a strategic priority. A customer initiative has already been promised to the market. Hiring is frozen. Several of the same architects, data specialists, cybersecurity professionals and change leaders are required by every program.

Each commitment may be rational. Together, they may be impossible.

Someone asks the AI assistant to recommend a plan that protects all of them.

The answer arrives almost immediately. It proposes phased delivery, tighter governance, selective automation, resource sharing, increased collaboration and a revised sequence. It sounds balanced. It may even sound reassuring.

But did the answer prove that the commitments can coexist? Or did it produce the most plausible story that satisfies the request?

That distinction matters.

In 2025, OpenAI rolled back an update to GPT-4o after concluding that the model had become overly flattering and agreeable. OpenAI described some of the responses as overly supportive but disingenuous and acknowledged that the model had been too influenced by short-term user feedback.

The point is not that AI cannot be trusted. The point is that an AI system can be highly intelligent, useful and well-intentioned while still being pulled toward the answer its user would prefer.

That is manageable when the stakes are wording or tone. It becomes dangerous when the question is whether the enterprise can afford, staff, sequence and deliver everything leadership wants.

A separation of powers

Organizations need a separation of powers for AI-assisted decision-making.

Human judgment should establish intent. Leaders decide what matters, which outcomes deserve protection, what risks are acceptable and which tradeoffs the organization is willing to make. No mathematical model can decide what a company ought to value.

Governance should establish authority. It determines who may change a priority, move a date, redirect capital, accept more risk or relax a constraint.

PMI’s 2026 Closing the Change-Readiness Gap report argues that enterprise agility depends on aligning intent, authority, structure and trust. Yet only 41% of executives surveyed believe their operating models support rapid allocation of capital and talent.

The ability to move resources quickly is important. So is the ability to see what that movement changes elsewhere.

Mathematics should establish feasibility. Once the assumptions, capacity, funding, dates, dependencies and constraints have been made explicit, the organization needs a protected calculation of whether its commitments can coexist.

Math does not decide strategy. It does not make imperfect data perfect. It does not remove politics, judgment or uncertainty.

It does establish where judgment ends and wishful thinking begins.

If 12 initiatives need the same six specialists during the same quarter, the organization does not have a communication problem. It has a capacity collision.

If a budget reduction removes the resources needed to achieve the original business case, the economics have changed even if the presentation has not.

If two regulatory commitments depend on the same release window, confidence will not resolve the sequence.

AI should establish understanding. It can question assumptions, find inconsistencies, identify patterns, propose alternatives and explain why an option succeeded or failed. It can help leaders ask better questions and explore complexity without waiting for days of manual analysis.

But the sequence matters.

AI may recommend that an assumption change. It should not silently change that assumption to produce a more acceptable answer.

Protecting the truth layer

CIOs are accustomed to protecting data. The next challenge is protecting the authority of different kinds of information.

A recorded fact is not the same as an assumption. An approved risk tolerance is not the same as an executive preference. A mathematically calculated shortfall is not the same as an AI-generated interpretation. A proposed option is not a commitment.

When all of these appear in one polished response, the distinctions can disappear.

EY’s analysis of the 2026 COSO framework makes this problem tangible. EY argues that control for AI-enabled decisions must move upstream, preserving evidence of the inputs, model outputs, human review, exceptions and changes that shaped the judgment—not merely documenting approval after the decision has been made.

That is a useful way to think about a protected truth layer.

The AI should be free to interrogate the facts, challenge assumptions, recommend alternatives and explain consequences. But changes to a date, budget, dependency, constraint or risk tolerance should remain visible, attributable and governed.

Otherwise, the enterprise may believe it is evaluating a new option when the AI has actually altered the question.

Speed makes the distinction more important, not less. West Monroe’s 2026 Speed Wins research found that more than 1,200 leaders reported losing up to 5% of annual revenue because decisions and execution move too slowly.

Organizations do need to decide faster.

But accelerating the conversation without protecting its underlying truth can simply produce a bad decision sooner.

What CIOs should require

Before allowing AI to influence major transformation or portfolio decisions, CIOs should be able to answer four questions:

1. Can the system distinguish recorded facts, governed assumptions, constraints, executive preferences, calculated results and AI-generated interpretations?
2. Can it explain which dependency or constraint made an option infeasible?
3. Can AI recommend changing an assumption without changing it automatically?
4. Can leaders reproduce the calculation and trace the recommendation back to the decisions and data that created it?

If the answer to any of these is no, the organization may have an intelligent conversational interface. It does not yet have a trustworthy decision capability.

This is not an argument for keeping AI out of the decision room. Continuous transformation may make AI indispensable. The volume of change, the number of moving parts and the speed of interaction across an enterprise are becoming too great for people to process unaided.

But AI cannot be the source of the facts, the interpreter of the facts, the judge of feasibility and the author of the recommendation without clear boundaries among those roles.

Humans must retain responsibility for intent and judgment. Governance must make authority and changes explicit. Mathematics must test whether commitments can coexist. AI should make the resulting complexity easier to explore, understand and act upon.

Continuous transformation requires a mechanism that can absorb a new condition, expose what it affects, test feasible responses and present credible options while the decision is still being made. AI can make that mechanism far more accessible.

It should not be allowed to make an impossible option sound possible.

As AI enters more executive decisions, the most important question may not be what the system can do.

It may be what the system is not allowed to negotiate.