Deconstructing the automatable decision

This article was co-authored with Zohar Strinka, principal consultant for analytics strategies and founder of meta-problem.com.

When organizations look for ways to use technology to drive efficiency, they often start with automation. Their business teams explain how data-driven they are, and CIOs believe them.

That belief is the first mistake.

The gap between what people call data-driven and what’s actually happening at the operational level is where automation projects quietly die.

Some tasks are perfect for automation. Think an analyst who generates standard reports every month or places purchase orders from suppliers based on reordering policies.

These tasks do not vary and typically don’t require human judgement. They are truly data driven. The problems arise when businesses try to automate processes that look like they’re data-driven, but that tacitly require human input to work properly. Recognizing which case, we’re in can be challenging.

The truth we don’t talk about enough is that truly data-driven decisions are rare. There is a range of ways that operational teams use data, and they are not always what they appear to be.

A CIO at a mid-size manufacturer was ready to automate their inventory replenishment decisions. Eight months in, the project had stalled. The data was in the ERP and the rules were documented. The team had described their process as data driven.

We found the planning team had been compensating for inaccurate supplier lead time data for years. They knew the system was wrong, so they never trusted it. Every decision ran through two people who carried the real picture in their heads. The automation had been built on data nobody used. The process wasn’t data-driven, it was data-ignored. 

In reality, decisions involving data tend to fall into one of three categories. Let’s take a closer look at these.

The decision-making spectrum

On one end we have truly data-driven decision making: A repeatable, scriptable process that takes the data as it exists in the system and transforms those inputs into concrete decisions. This kind of process is easy to automate, and the outcomes are exactly what you’d expect, resulting in high quality decisions every day.

In the middle we have data-informed decisions: Where those in operations use data, conduct analysis and use judgement to ultimately decide what to do. Automating a data-informed process is tricky because it requires the current decision makers to put into words how they transform raw information into actions.

Most organizations fall into a third category of data-ignored decisions: The right action depends on information that is flat out wrong in the system. The spike in sales which was driven by a special order. The supplier lead times that haven’t been updated in years. The customer-specific pricing that automatically gives a discount they didn’t earn.

If CIOs push ahead with automating decisions that are currently data-informed or data-ignored, they risk investing heavily in tools that will never come close to human performance. Humans often know the data is wrong, or when the rules don’t apply, so they override the data-driven outputs.

That’s not dysfunction, it’s judgment.

The real cost of getting it wrong

Organizations that rush into automation without understanding how decisions are actually being made don’t save time. They build systems that don’t work, retrain teams to work around them and eventually either scrap the project or live with a system that has 100% human review in practice. That’s not automation, its expensive theater, sometimes to the tune of 10x the original budget, with nothing to show for it but a system nobody trusts and a team that’s learned to work around it.

The discovery work – mapping the data, sitting with the team, deconstructing the decision, is investment not overhead. Done properly upfront and the automation that follows is faster, cleaner and actually sticks.

The 10X isn’t what good automation costs. It’s what bad automation costs you later.

When executives plan for the real cost of discovery, only the highest-value automation projects clear the ROI bar. That’s a feature, not a bug. It leaves the technical team focused on the work that actually moves the needle.

How to succeed with automation

There are four key steps to moving from an existing manual process to automated decisions. Work through each step and you’ll automate the right decisions instead of building something that doesn’t work in practice.

Step 1: Identify where the data actually lives

With a plethora of systems, it’s easy to think that the data you need to drive a decision will be available and accessible. Unfortunately, organizations have nuances that make the vision of a one-stop-shop ERP a fantasy.

Begin with an inventory of the information needed to make the decision you’re trying to automate. AI can be a valuable tool to help systematize access to that information. Check if the systems have all the needed info or if there are spreadsheets or knowledge in people’s heads that need to be centralized. You can’t automate what you can’t locate.

Even when all the data does exist in systems, it can be hard to pull it into a form that works for automation. Don’t underestimate the crucial infrastructure steps to pull it all together.

Step 2: Go deep with the team

Most how-to guides suggest automation is as easy as pulling together the data, hooking up the systems and automating exactly what the team says the decision-making process is.

However, since data-driven decisions are rare, the real next step is to work shoulder-to-shoulder with the team to map all aspects of the process. How do they know which decisions they need to make, and when? What information do they use, and how do informal signals play a role in their process?

When you take the time to explore with the team, you often discover surprising challenges. In one project we saw how counterfeit products in ecommerce could instantly crater sales. It would have been easy to respond to that competition by lowering prices, when in reality the right decision was to act against the seller.

Another common failure point in automation projects is the data that is sometimes incorrect because of the people and systems that generated it. The category which is almost always whatever the first value in the drop-down menu is. The requested delivery date which is simply a week from when the order was placed.

We often discover that the existing decision-making processes are severely limited. One organization was using a simple 30-day rolling history to decide how much inventory they needed. Automation was a chance not only to increase efficiency but also improve the quality of the decisions.

Step 3: Build the decision model

The key insight here is that what looks like a single decision is almost always a sequence of smaller decisions stacked on top of each other.

Take network planning at a large milling company. On the surface the decision looks simple: Where should we produce this order?

But underneath it breaks into at least five distinct choices-  which facilities have available capacity, which routes minimize cost and transit time, which customer commitments are at risk if we shift production, which quality specifications constrain the options, and which combinations of the above are still profitable. A human planner navigates all five in their head, often in minutes. It looks like one call. It’s actually five.

Building the decision model means making that sequence explicit. Each sub-decision needs its own logic, its own data inputs, constraints and its own threshold for when to escalate to a human. Only when you’ve mapped the full sequence do you know what you’re actually automating and where the judgment still needs to live.

The decisions most worth automating are consistent, frequent and complex enough that no human could match the speed or scale of an algorithm. But you can only get there after the deconstruction.

Step 4: The iteration reality

The most important question is what not to automate because a human can do it better.

A robust decision model will still automate the easy cases. PayPal famously implemented a red flag system where most transactions sailed through without issues. The ones that seemed potentially fraudulent were routed to a human. By automating the majority, people could quickly act on the few leftover cases that required judgement.

A food manufacturer we worked with was processing thousands of daily inventory replenishment decisions across a national distribution network. On paper, the rules were clear. In practice, their team was manually reviewing nearly every output the system generated.

When we went deep, we found the problem. The system had no way to flag the exceptions. The supplier who was unreliable during harvest season, the SKU with a promotional spike coming, the warehouse running at 95% capacity. So, the team reviewed everything, just in case.

The fix wasn’t better automation. It was building an exception library, a growing catalog of the scenarios that broke the standard rules. Once those were coded, 80% of decisions ran automatically. The remaining 20% were flagged with context, routed to the right person and resolved in minutes instead of hours.

The iteration reality is that you don’t automate everything. The goal is to automate what’s safe to automate while making the exceptions visible, fast and actionable for the humans who need to act on them.

What good looks like

Efficiency is the easy part of the story. The organizations that get decision automation right don’t just move faster, they make fundamentally better calls, at scale, consistently, with a record of why. That shows up in margins, in customer commitments met and in operational resilience when things go sideways. So often automation projects fail because of all the system and human steps that go into success.

Said another way, there are about a hundred ways to automate a bad decision, and only a handful of ways to automate a good one.

Investing in automation the right way changes the operational layer.  Better decisions get made faster, more consistently and are auditable

The CIOs who get this right are the ones who resist the pressure to move fast on a foundation they haven’t examined. The question was never whether to automate — it was whether the organization understood its decisions well enough to automate them responsibly. Most don’t, yet. The ones who do that work first will build operational advantages that are genuinely difficult for competitors to replicate.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?