Enterprise IT moves forward — cautiously — with generative AI

Aside from the benefits, CIOs deploying the technology need to be aware of potential intellectual property issues regarding generated outputs, CarMax’s Mohammad cautions. Generative models, such as DALL-E that trains on data from the Internet, have generated content that may infringe on copyrighted content, which is why Getty Images recently sued Stability AI over its AI-driven art generation tool Stable Diffusion.

Mitre Corp.

The technology also needs human oversight. “Systems like ChatGPT have no idea what they’re authoring, and they’re very good at convincing you that what they’re saying is accurate, even when it’s not,” says Cenkl. There’s no AI assurance — no attribution or reference information letting you know how it came up with its response, and no AI explainability, indicating why something was written the way it was. “You don’t know what the basis is or what parts of the training set are influencing the model,” he says. “What you get is purely an analysis based on an existing data set, so you have opportunities for not just bias but factual errors.”

Wittmaier is bullish on the technology, but still not sold on customer-facing deployment of what he sees as an early-stage technology. At this point, he says, there’s short-term potential in the office suite environment, customer contact chatbots, help desk features, and documentation in general, but in terms of safety-related areas in the transportation company’s business, he adds, the answer is a clear no. “We still have a lot to learn and improve to be able to include generative AI in such sensitive areas,” he says.

Jeter has similar concerns. While his team used ChatGPT to identify a code fix and deploy it to a website within 30 minutes — “It would have taken much longer without ChatGPT” — and he thinks it’s useful for drafting terms and conditions in contracts, it’s not entirely proven. “We will not expose any generative AI to external members,” he says. “TruStone will not be bleeding edge in this space.”

TruStone Financial Credit Union

When TruStone eventually starts using the technology for the benefit of its members, he adds, it will monitor conversations through human and automated review to protect its members and the brand.

Today, the key to successful deployment is still having a human in the loop to review generated content for accuracy and compliance, says UCSD’s Kellen. “Making sure the machine makes the right decision becomes an important litigation point,” he says. “It’ll be quite a while before organizations [use it] for anything that’s high risk, such as medical diagnoses.” But generative AI works fine for generating something like review summaries, provided there’s a human overseeing them. “That slows us down a bit, but it’s the right thing to do,” he says. Eventually, he adds, “We’ll find automated ways to ensure that quality is good. But right now, you must have a review process to make sure the content generated is accurate.”

UCSD

Another well-documented risk, in addition to accuracy, is the potential for bias in the models introduced from the data used to train them. This is especially problematic when generative AI is using content from the Internet, as ChatGPT does, but that may be less of an issue when training the model against your own private corporate data that you can review for potential bias, Kellen says. “The more you get to the enterprise, where the class of data is more constrained and more mundane, the more generative AI shines,” he says.

The thing to understand about large-language models, says Cenkl, is these machines are to some degree savants. “They don’t understand, but they’re very good at computing,” he says.

Changes in job responsibilities, roles

“Technology has made things better, but it’s also created a lot of extra work for us,” says Mohammad. However, he believes generative AI is different. “It’s exciting because it’s going to take away some of the stuff we don’t like to do and make us more intelligent,” he says. “It will augment humans.”

But Curran points out that there’s no expectation that generative AI will completely replace any role in the short term. “It may reduce the number of people needed to execute a role, such as in content development, product information management or software development,” he says. “But there will always be the need for a human in the loop.” And Mohammad adds that even if the technology can write and summarize, human intelligence will always be needed to ensure quality, and to control what’s been generated to make it better.

Steps to get started

Now is the time to get up to speed on generative AI technology and start experimenting, says Kellen. “CIOs have to get their heads inside this puzzle before they’re bamboozled by vendors who are embedding the technology into their enterprise software offerings,” he says. “If you spend the next year procrastinating, you’ll be behind the curve.”

It’s important to get educated and go deeper than the public discussion on ChatGPT in order to understand that this technology is much more complex than one application, says Curran. Then start considering use cases where generative AI might improve the efficiency or quality of existing processes. Finally, ask what types of capabilities you’ll need and whether you should acquire that from a vendor or build it yourself.

From there it’s a matter of testing the technology and consider potential use cases. “A lot of your systems, whether they use structured or unstructured data, will have at least some component of natural language and conversational interface,” says Cenkl. “Think about the data you have and what parts of that can be augmented by these technologies,” and then demonstrate the potential. For example, Jeter says he generated a template of terms and conditions and sent it to his compliance department to show how they could use it.

Generative AI models are large, and training them from scratch is expensive, so the best way to get started is to use one of the cloud services, says Curran. CarMax, for example, uses Microsoft’s Azure OpenAI Service with GPT 3.5. “The data we load is our own — it’s not shared with others,” Mohammad says. “We can have massive amounts of data and process it very quickly to run our models. If you have a small team or business problem that might take advantage of generative AI technology, give it a shot.”