The best pilot project might be the one you kill

One perennial issue IT leaders face is that project success rates remain less than ideal. For example, survey data from recruitment firm Wifi Talents showed that, in 2023, fewer than 30% of software projects were completed successfully, and that 75% of IT and business executives anticipated that their projects would ultimately fail.

Figures like these are sobering. And they are a key reason why many IT organizations choose to initiate pilot projects, especially when engaging new or emerging technologies, first before jumping into deeper water.

Pilot projects are small-scale test runs aimed at determining the feasibility of a project before full-scale commitment. They give IT leaders insights into potential roadblocks and opportunities, and they help IT leaders make informed decisions on whether to proceed with a full-scale version of the concept by giving them practical feedback on the timeline, resource, cost, and outcome implications of doing so. 

Companies of all sizes use pilot projects because they are a great way to test new technologies and systems with a minimum investment — and with an upfront understanding from the board and other key stakeholders that the project might not work.

“We use pilot projects all of the time,” a manager at a large AI database company shared with me. “And if they don’t start showing results in six to eight weeks, we pull the plug.”

Ideas like pulling the plug and minimizing investment are fundamental to a sound pilot project strategy. But at what point should IT leaders expect toget viable results from their myriad pilot project efforts?

Pilot project timelines

Ninety days seems to be the de facto time frame given to most pilot projects, but there is no hard and fast rule. This is because time frames depend on the nature of the pilot project you are running.

If you’re testing a new financial workflow that will shorten the month-end close by one or more days, the pilot project will minimally need to run for one month, and possibly for even an extra month or two to ensure that month-end results remain consistent.

If you’re programming a robot to pick parts and report progress to a central warehouse system, you might only need four to six weeks to confirm whether the proof of concept is viable.

In both cases, you must define in advance very specific metrics and outcome expectations that the pilot must produce to be considered viable. You must also set the timeframe in which results (good or bad) are expected. And you must communicate the project, timelines, and expectations to the pilot project team, the board, and management stakeholders. You’re your timeframe expires, results can be reviewed, and a decision to move forward or terminate can be made.

Avoiding pilot project ‘sand traps’

Estimating the timeline and metrics for a pilot project is key — and can often be more art than science. It’s understandable as well that emotional stakes may be involved in a pilot project, whether from the IT leader, board, or CEO, that could influence objectivity when it comes to deciding when to pull the plug.

Still, while pilot project methodology can sound deceptively straightforward, there are several other “sand trap” challenges that pilot projects must navigate, including:

Low expectations

With pilot projects, you don’t have those mad pushes to launch critical applications into production. In fact, most people perceive pilot projects as “just kicking the tires.” Attitudes like this can manifest as a de-prioritization of work on pilot projects, which can ultimately lead to stagnation in pilot project work and timelines.

Failure to declare failed pilot projects DOA

When I visited with the manager at the AI database company, he told me that one of the most important things that the company did to stay on track with innovation was to aggressively declare pilot projects as “over” as soon as the projects showed they would not work. This wiped failed projects from the slate so that new areas of innovation could be pursued. So many companies forget to do this. They keep pilot projects on the docket for years — until someone asks whatever happened to pilot project XYZ?

Failure to communicate and agree

People don’t always agree on pilot project selections, so it’s important to know before you start whether there is enthusiastic backing for the project.

My IT group once had an idea to shorten the timeline for processing employee payroll, which Finance thought was great, too. Together, Finance and IT calculated how many hours and costs would be saved by the automation we were proposing. We felt our workflow revision would work, but we wanted to pilot it first. The problem was: No one in the company except for IT and Finance even cared.

The argument we got was that the pilot project returns were only “soft” savings of dollars and time. The Finance staff would only reallocate time and hours to perform other work. Our upper management critics didn’t feel that the project was wrong or that it wouldn’t work. They just didn’t see it being that important.

We didn’t do this project, and we were glad we didn’t, because there was no buy-in.

Bottom line: Be decisive

Of the hundreds of companies and CIOs I visit with each year, I can think of not one that doesn’t have at least one pilot project stuck in the weeds because there isn’t time to work on it, or people have lost interest.

Rather than allow these pilot projects to linger, it’s best to be decisive and lose patience with them when they don’t produce the results intended within the timeframes declared.

Having also sat on boards of directors, I can share this insight: Board members want to see their companies moving forward, and running pilot projects is central to that. But they also appreciate hearing about projects that are cancelled because they don’t work. That’s because they want to know that the officers within the company are every bit as concerned about returning value from investments as they are.

And that includes investments in testing out the next big thing.