I audited an award-winning AI project. The case study left out the cloud bill

I have audited enough celebrated technology rollouts to notice a quiet omission across the industry: nobody talks much about what happens to the unit economics once the pilot ends and the software enters daily production.

I have anonymized the company and changed identifying details for confidentiality, but the operating economics and workflow behaviors below are taken directly from the post-implementation audit.

About six months ago, I was brought into a mid-sized enterprise to review a document workflow that had recently received an industry innovation award. The platform vendor published a case study highlighting the deployment. The trade press ran a short feature on it. Internally, the business unit sponsor pointed to a ninety percent reduction in turnaround time for vendor onboarding agreements, using that headline metric to secure a larger operational mandate for the coming fiscal cycle.

A couple quarters later, the finance team reached out with an uncomfortable question. Document volume had stayed flat, yet the monthly operating expenses attached to processing those agreements were climbing steadily every single billing cycle.

They asked me to trace the spending back through their systems and reconcile the operational data with the general ledger.

Under the previous manual workflow, an operations clerk opened an incoming contract, checked five or six key clauses against internal guidelines, verified the vendor details and filed the document away. The entire review took about four to five minutes. Using the department’s fully loaded labor rate, including benefits and allocated overhead, that came out to roughly eighty cents per document.

The automated pipeline was running between $12 and $14 for the exact same file.

The system was technically stable. It processed thousands of files a month without dropping connections or crashing servers, and the operations dashboard stayed green. But the business was spending $12–$14 to complete work that had previously cost about 80 cents.

Every enterprise award celebrates speed and throughput. Few of them audit the cost to finish a single unit of work.

Nobody caught the discrepancy during the original three-month pilot because the computing bills, data lookups and third-party model access fees had been charged to a central innovation fund. The departmental budget never saw an invoice. As far as operations was concerned, the tool was essentially free software that dramatically reduced turnaround times on incoming requests.

The moment the pilot concluded and the system moved into full daily production, the accounting changed. Corporate finance closed the innovation project code and began allocating the actual infrastructure invoices directly back to the business line. That was when the operational reality surfaced. Within sixty days of running full transaction volume on the department’s budget, the economics of the unit turned negative.

Traditional enterprise software often has a predictable cost curve. Once the baseline platform is running, processing another five hundred records usually does not require the same kind of fresh computation and external model consumption as an automated pipeline.

In this kind of workflow, each customer interaction, incoming contract or automated data extraction can trigger additional computational steps. The system had to interpret the document, check it against internal rules, compare the result with reference data and validate the answer before completing the transaction.

If an automated pipeline requires multiple model queries and several background checks to finish a task that an experienced employee used to resolve in five minutes, you have not eliminated an operational cost. You have swapped a predictable human payroll line item for a consumption-based cost that scales directly with the work.

Falling model prices don’t mean falling workflow costs

The standard counterargument is that computing power is getting cheaper. That part is true. Research published in the Stanford HAI AI Index Report documented that the cost of querying benchmark foundation models dropped by more than 280-fold between late 2022 and late 2024.

It is tempting to assume that automated business processes will experience the same cost reductions.

In practice, a dramatic drop in per-token rates does not guarantee a lower operating bill if the production workflow performs significantly more retrieval steps, document parsing passes and validation checks to complete a single business task.

Gartner’s latest forecast points to the same tension, predicting that inference costs per agentic workflow will increase more than fivefold through 2028 as increasingly complex workflows offset falling model prices.

Both things can be true at the same time: base model prices can fall while the total cost of completing a business transaction rises.

Cheap models do not guarantee cheap business processes.

The economics of an automated system depend on the cost of the entire workflow, not just the base model.

Production complexity is where the bill gets bigger

During our audit, the operational problem showed up the moment real-world documents stopped looking like the clean samples used during vendor sales demonstrations.

The demonstration files were digitally generated PDFs with uniform fonts, consistent layouts and standard contractual language. The live production queue was full of low-resolution scans, photocopied forms rotated sideways, documents with handwritten marginal notes, and agreements containing conflicting payment terms.

A human operations specialist can often resolve that kind of ambiguity with a quick judgment call based on institutional context. The automated workflow struggled when the document fell outside the assumptions built into the process.

When the system encountered messy formatting or contradictory clauses, it triggered extra processing routines. It ran multiple extraction passes, queried internal reference databases and executed secondary verification checks to resolve the ambiguity. Each extra step added to the cloud invoice, yet the system still struggled to reach an acceptable confidence score.

When confidence fell below the required threshold (which occurred in roughly forty percent of daily transactions), the pipeline halted and routed the file into an exception queue.

That created a secondary labor expense the original business case never budgeted for.

When an agreement landed in the exception queue, an operations specialist had to log in, read the system’s partial extraction notes, diagnose why the software stalled and manually enter the correct data into the core database. Because the specialist had to review both the original document and the system’s confused output, resolving an exception took ten minutes (twice as long as the original manual baseline).

The company found itself paying twice. It was paying the full monthly cloud invoice for the automated software, while simultaneously retaining the payroll cost of the operations staff required to supervise the exceptions.

Goldman Sachs has raised the broader macroeconomic question of whether the massive corporate spending on AI infrastructure will produce productivity returns sufficient to justify the investment. Our audit showed what that question looks like inside a single department: replacing an eighty-cent clerical task with a fourteen-dollar automated pipeline that still requires human labor for four out of every ten transactions.

The problem rarely comes from the user-seat price on the purchase order. It comes from what the system consumes to complete the work.

The vendor can price the platform around usage, but the customer owns the business case. If an automated pipeline requires multiple processing passes to interpret a crooked document scan, the customer still has to absorb that consumption whether the final output is usable or not.

When you scale a human operations team, the additional cost is relatively predictable. You know what another specialist costs in wages, benefits, equipment and management time. For this kind of workflow, automated costs can rise with the ambiguity of the inputs.

When the fiscal quarter wraps up, this creates an organizational disconnect where every department claims victory while the business loses margin:

• The engineering team reports ninety-nine percent system uptime and clean infrastructure performance.
• Operations leadership reports that turnaround times on clean files dropped from forty-eight hours to under a minute.
• The software vendor references the client in marketing collateral as evidence of enterprise transformation.
• Finance is left trying to figure out why the cost of goods sold increased while revenue stayed flat.

When leadership asks why an automation milestone fails to improve operating income, engineering points to platform uptime, product leads point to system adoption and software vendors point to successful transaction counts. Every department hit its individual performance goals. Yet the business process itself became strictly more expensive to run than it was before the project started.

In this deployment, the team tried to fix the issue by adding secondary filtering scripts and tuning prompt instructions. Those additions increased system complexity and cloud consumption without solving the core unit cost problem.

An automated workflow can execute with zero technical errors while still undermining departmental margins.

Three questions before you call an AI project a success

Before approving a production, rollout or expanding an automation pilot, finance and technology leaders should put three direct questions to their teams:

• What does one completed transaction actually cost compared with the manual baseline, accounting for all infrastructure compute, database lookups and third-party model charges?
• How often does a human still have to intervene, and what is the loaded payroll cost of that review time?
• Do the economics still work at production volume, or is the organization simply moving an operational expense from payroll into an unpredictable consumption meter?

If the economics only work during the pilot, the system is not creating operational leverage. It is creating an operating expense the pilot never measured.