Why we said no to AI chatbots

During an impromptu meeting, my boss said, “We need to look at deploying an AI chatbot to help us save money. Executive leadership is very interested in this.” I was ready. After years of building a foundation of operational excellence, I replied, “They’re all more expensive. And the one that isn’t has a four-year ROI and degrades our ability to support the business. Here’s an executive summary slide for the executive team.”

Is the goal to provide support faster and cheaper or is that fixing a symptom? 

For me, the answer to this question came as a result of 10 years leading IT employee productivity services and being significantly involved in the adoption of a product-aligned organization, where IT was structured in a similar manner to a traditional product function. Or as I view it: running IT like a business. These experiences spanned both public and private equity-owned software companies undergoing transitions to subscription models. Operational excellence and financial efficiency — to put it politely — are table stakes in a private equity setting and laid the foundation for why AI chatbots didn’t make the cut.

Operational excellence isn’t flashy, it’s foundational 

In 2015, I joined a publicly traded SaaS company, where I spent most of my time leading collaboration services and delivering what an executive leader described as “radical collaboration” for a global company of 11,000 employees. This was due in part to supporting the transformation to, and leading by example, the adoption of a product-aligned organization and running my services like mini businesses. Moreover, I leveraged my graphic design and MBA educations to market and sell my services creatively.

It’s also where I coined a mantra: “No instructions!” as a means of truly embracing a frictionless experience for employees using IT services. Why should they need instructions for our services when billions of people buy smartphones, and they don’t come with instructions? Yes, if someone wants to do something beyond the basics, they may require help and documentation, but again, that’s no longer basic support — it’s a corner case.

The results that ultimately ruled out AI chatbots came from my time in a private equity-backed company that I joined in 2019, where I ultimately led IT Employee Productivity services, spanning a large portion of the IT infrastructure organization. Within weeks of joining, I discovered our L1, L2 and L3 teams were overwhelmed by support tickets. I remember asking my most senior O365 engineer/architect how he spent his time. His reply shocked me; 80% of his time was doing L1 and L2 tickets! This, coupled with multiple major in-flight initiatives, was a scenario that would not be tenable.

As a result, I realized this would require a two-part, simultaneous move—jumping from reactive firefighting to a strategic, product-aligned model in one coordinated move. One move involved aligning teams to product/service ownership, giving them A–Z ‘accountability’ as described by the RACI model. The second would be getting the right people doing the right work by optimizing the support of our services. For the sake of this article, I will focus on the latter, a shift-left initiative that, in essence, is the moving of tickets to the left from L3 to L2 and L2 to L1. 

The immediate goal was to free up engineering resources so they could, in turn, focus on eliminating additional root causes of tickets. Operationalizing this meant restructuring our monthly service review (MSR) to focus heavily on ticket reviews. We turned it from a readout meeting where people multitasked into a working session for driving discussion and gaining alignment. This allowed us to prioritize root-cause remediation and service simplification and then allocated sprint capacity to the ‘low-hanging fruit’.

In the beginning, very little sprint capacity was available, but not surprisingly, people found time for work that ultimately made their lives easier. This allowed our engineers to reduce ticket volume at the source. It wasn’t glamorous, and it wasn’t easy, but it was strategic.

Over two years, we cut L1 tickets by 55% and total ticket volume by 29% (adjusted for staffing changes). L3 engineers/architects were focused on strategic initiatives, innovation, architecture, security and mentoring. There was an added benefit of creating career growth opportunities for the L2 teams as they were getting time to work with L3 engineers on projects. Moreover, we were able to reduce our L1 managed service provider (MSP) spend that provided our offshore IT help desk. Additionally, we worked with them to evolve their delivery model and they were very accommodating to support our shift left initiative, whereby allowing us to up-level their staff beyond basic IT Support and ‘amend’ our contract terms with them.

But we didn’t stop there. We cleaned up a bloated knowledge base within ServiceNow, pruning outdated documentation and rewriting what remained with clarity and user experience in mind. We brought the same data-driven approach to the KB reviews as well as my mantra, “no instructions needed.” Articles with high view counts were evaluated for service enhancements that could eliminate the need for the article in the first place.

As a culture of product ownership and pride in operational excellence took hold, more ideas started coming from the teams. A self-serve asset return function, cut 6% of the L2 ticket workload, and training IT staff on communication best practices via ServiceNow reduced ‘ticket status check’ calls by 70%. And most notably, the announcement of a pending merger created the opportunity to align with the business on the need to completely reinstall the operating system on 2,000 machines, about 30% of the workforce.

A previous deployment of laptop auto-provisioning meant the whole process took 3 hours per user on average and was self-serve in many cases, enabling us to complete this otherwise massive effort in just 10 weeks. The result was a dramatic reduction in password reset tickets as our self-serve solution now worked globally with no caveats.

With a solid foundation in place and the team feeling good about how we were doing, I asked the team how we stacked up against industry peers. This resulted in my leading an effort to further optimize our IT Support services through the lens of time, quality and cost. We looked at these metrics across IT support tiers and by service.

This illustrated pockets of IT support that were still very expensive and clearly not optimized. Most notably, certain sites, such as Singapore and our HQ in the San Francisco Bay Area, where IT support was heavily built around informal relationships with on-site IT support. Additionally, our in-person walk-ups were extremely expensive as they were manned by the most senior of our L2 techs. In both scenarios, we had senior L2 IT support doing primarily L1 ticket work.

Changing this was not easy, as it was ultimately a culture change; however, we were successful. We built the case to limit tech stop hours based on optimizing around utilization; we would be open when people show up the most. We coupled this with signage and marketing of our L1 IT service desk, highlighting specific tickets they resolved quickly and effectively. This was done through email, posters, digital signage and most importantly, word of mouth from the L2 IT support staff.

AI chatbots weren’t the solution, because support wasn’t the problem

Then came the AI revolution. Execs asked whether we could replace our L1 MSP with an AI chatbot solution. To executive leadership’s dismay, my one-page summary slide made the rounds, showing it was more expensive. And for the one vendor that showed a positive ROI, it was more than four years out and came with significant downsides as we’d lose manpower for supporting big initiatives, cutovers or hyper-care needs that our MSP routinely provided.

This was my lightbulb moment: AI isn’t fixing the problem, it’s addressing a symptom of poorly designed IT services by improving support efficiency metrics. Specifically, AI chatbots are reducing ticket costs and improving a few support efficiency KPIs associated with being better and faster at regurgitating KB and FAQs than a human being. This isn’t a revolutionary or transformational shift; it’s incremental progress in delivering IT support.

One could argue that if AI chatbots are dramatically improving the IT support experience, maybe the underlying IT services are not being deployed with enough attention to the user experience of those services. And inevitably, in a few short years, the same questions will come up again: how can we save money in IT support and why are our employees so unhappy with IT’s services? Because ultimately, the real issue isn’t support — it’s service design.

For larger organizations, AI chatbots may have a place, especially as part of a well-architected support strategy dealing with high ticket volume. But for most mid-sized enterprises, the AI chatbot ROI falls apart when IT services are designed to truly be frictionless and IT support is truly optimized.

In our case, fixing the underlying problem was not easy. It was a multi-year journey that took a lot of strategic planning and constant effort to ensure it stayed prioritized amid competing initiatives, including aggressive cost-saving mandates in a PE environment. In the end, the best AI chatbot strategy was one we didn’t need, because we had already solved the real problem. 

This article is published as part of the Foundry Expert Contributor Network.
Want to join?