CIOs worry about Gen AI – for all the right reasons

Many CIOs are wringing their hands over generative AI. No, the apocalyptic visions of the groundbreaking new technology replacing us – even destroying us – aren’t keeping them up at night.

Rather, they’re worried about how best to arm their employees as quickly and safely as possible with what could turn out to be the most consequential information technology of the decade. Generative AI chatbots like OpenAI’s ChatGPT are emerging as the ultimate no-code content-generation tools, with the capability to empower virtually any employee to produce drafts of budgets and customer proposals – even advertising jingles and presentation art – in just seconds. Many of the chatbots can even generate code on the fly in programming languages like C++ and Python.

Do this right, and CIOs can help their organizations leapfrog competitors by dramatically improving operations, streamlining marketing, and ratcheting up customer service. Fumble the opportunity and, well, that’s what all the handwringing’s about.

“CIOs don’t have to panic this month because everyone is just starting out,” said Shaown Nandi, Director of Technology at Amazon Web Services. “But if you don’t put a plan in place to incorporate generative AI, then in two to five years you will be behind. Your products, your solutions will be more limited. And you will have the slowest, worst-performing call center.”

Most commonly, organizations developing plans for generative AI are opting to fine-tune third-party models like OpenAI’s GPT 4.0, LLaMA from Meta, Google LaMDA, or Amazon’s Titan series, with their own proprietary data. As well, many enterprise data platforms are adding generative AI front ends to make their services more accessible and valuable.

Both options make generative AI much quicker and far less expensive to implement than building your own foundation models from scratch. They also offer the potential for better precision, privacy, and security than publicly available generative AI chatbots like ChatGPT, Google Bard, or Microsoft Bing.

“Every single vendor that we touch is baking generative AI into their product,” said John Musser, Director of Engineering for Ford Pro, Ford Motor Company’s commercial business. “So how do we best leverage that and put our own Ford Pro special sauce on top?”

Naturally, what you’re able to do – and how much risk that involves – depends at least as much on the state of your own enterprise data platform.

“Your data platform is the foundation for foundation models,” says Ram Venkatesh, Chief Technology Officer at Cloudera. “If your platform isn’t ready to enable more people in your organization to do more with your data, then you’re not ready for generative AI. It’s just that simple.”

Indeed. As CIOs get their arms around the myriad pitfalls of handing the company’s data jewels to a creation engine built by someone else, governance quickly rises to the top of the priority list.

As well, most organizations are also containing the potential for damage by confining generative AI models to support roles. Which means that no model will have the power to make decisions. And they won’t be able to interact with customers. For now, at least.

May I help you?

One of the most exciting aspects of generative AI for organizations is its capacity for putting unstructured data to work, quickly culling information that thus far has been elusive through traditional machine learning techniques.

That makes the technology a great fit for customer service and support, which are typically challenged with making sense of mountains of unstructured data, from records of customer interactions to training videos. In general, the more involved and complex the decision matrix, the greater the potential for improvement.

Like in healthcare, says Charles Boicey, Chief Innovation Officer at Clearsense, a data-as-a-service platform provider for healthcare. Among other big advantages, he sees generative AI models busting through the very manual process of creating chatbots.

“We had a team of people writing new responses for questions that came in that we hadn’t yet thought about,” Boicey said. “We were building responses to every damn question. Large-language models will eliminate so much of that, because we’ll be able to get perfectly good responses from the AI.”

And in insurance, according to Alex Cook, Head of Strategic Capabilities at New York Life Insurance Co. The insurer is developing an AI-based tool to help customer service reps better field wide-ranging questions about complex issues. Like, for example, a client question about a rider to a 35-year-old policy the company no longer offers – but is nevertheless still in force.

“Getting answers to questions like that isn’t easy, right?” Cook said. “Often, you’ve got service reps that weren’t even born when those products were issued.

“Quite often today, they’ll have to put the client on hold or call them back,” Cook continued. “It’s not very efficient and not a great experience for the customer. The intent is for these generative AI tools to allow that to be a one-and-done type conversation with rapid response.”

And Ford Pro, a business that provides telematics services on top of fleet vehicles and EV chargers, is building an LLM-powered chatbot to provide internal teams with faster, more accurate access to documentation.

The end-state for no-code

Democratizing AI has been the rallying cry of no-code/low-code tools for some years now, and generative AI is taking the concept to another level. With a chatbot backed by generative AI, insights normally confined to the province of data scientists and business analysts can now be within reach for anyone given access.

Clearsense, Ford Pro, and New York Life are all building out that capability. With good reason.

What’s most exciting to Cloudera’s Venkatesh is not just that more people can pursue answers from the data. As well, more data is accessible than ever before.

“The hardest type of data to make sense of has been unstructured,” Venkatesh said. “But it’s critical. What was the customer’s experience? Did we solve the problem? How many times did they have to call? And why did we not see this sooner? So much of that is hidden away in the chat history, not all the rows and columns of structured data.

“You don’t have to teach models anymore that loan and mortgage can be used interchangeably in some contexts. LLMs pick that up on their own. So the cost of extracting semantic meaning is probably 100 times less expensive than it was even 18 months ago. That’s huge.”

Ain’t representin’

For now, at least, most see generative AI as a tool to speed human decision-making and interaction. If anyone’s developing gen AI-infused applications now to make decisions or speak directly with customers, I haven’t found them. Surprisingly, though, folks are pretty divided as to whether they think that day will come at all.

Ford’s Musser and Cook from New York Life believe that likely will happen for some applications, once guardrails are locked down and the possibility of hallucinations is all but eliminated. Boicey from Clearsense isn’t so sure.

“We don’t fully understand how, cognitively speaking, humans come up with responses,” he said. “So we’ll invariably miss out on some types of inputs and conclusions. Call it intuition. Call it experience. Whatever causes a human to contemplate the gen AI response and say, ‘Yeah I get that, but it’s not the right call for this patient right here.”

AWS’ Nandi agrees, saying that the potential cost of handing over the keys outweighs the benefits.

“Particularly for a regulated industry, you’ll need to have really good guardrails and controls in place,” Nandi said. “And by the way, one of the easiest guardrails you can implement? A smart person.”