Tech leaders weigh in on the upside and flipside of generative AI

Generative AI is widely regarded as one of the great technology breakthroughs of our time. On the back of thousands of headlines provoked by OpenAI’s ChatGPT, it’s provoked urgent responses from many tech giants and is the theme of, and main topic of discussion at, tech conferences worldwide. But, as with any big new wave, there is a risk of once-promising projects being washed up and there are clear and obvious concerns over governance, quality and security. To cut through the froth, CIO.com polled a range of IT leaders and experts for their views on where we are with generative AI, their hopes and their concerns.

The state of play

The storied UK IT chief Paul Coby, now CIO of family property developer Persimmon Homes, has seen many trends come and go but he is bullish on generative AI, even though it only made its first appearance in November 2022.

“I believe generative AI is a game changer at a fundamental level,” he says. “I was at a Gartner conference in the US where they called generative AI out as the ‘third digital revolution’ after mainframe computing and the internet. The impact could really be that profound since we have a tool that can be applied to multiple use cases, from writing and designing products, to visualizations, checking code, and so forth.”

Another experienced IT leader, David Ivell, chief product and technology officer at behavior management training company Team Teach, is already harnessing generative AI’s power.

“Generative AI is a key part of our business strategy, facilitating growth with AI-enabled processes already live in production,” he says. “Since the middle of last year, we’ve been analyzing the potential impact, opportunities, and risks of the speed of innovation in this area, as well as introduced policies and implemented measures to minimize risks,” he says. “But overall, we see this as a huge opportunity. We ran workshops with every division of our business, educating them on the accelerating innovation in this area, brainstorming opportunities and risks. We’ve been shortlisting and building out potential proof-of-concept options and modelling revenue impact and already taken one concept through our innovation lab and into live production.”

Jon Collins of technology analyst firm GigaOm and author of The Technology Garden: Cultivating Sustainable IT-Business Alignment, is both a market watcher and user.

“We’re testing ChatGPT at an individual level,” he says. “The potential is highly positive and impactful, particularly as a research tool or one which gives an initial, albeit fully formed, answer. But it’s still to be seen how generative AI replaces, rather than augments, human involvement in terms of information. From a design perspective, such tools are more compelling.”

Neil Ward-Dutton, VP, AI and Intelligent Process Automation European Practices at IDC, suggests that generative AI usage is high but business strategy may lag.

“Generative AI has colossal potential to impact multiple areas of business,” he says. “An IDC survey from March 2023 saw 21% of respondents say they’re already investing in generative AI this year, and a further 55% are exploring potential use cases. In general, we see a small number of organizations using generative AI based on a strategy or plan, shaped by clear policies, and a lot of grassroots experimentation, but that’s almost always happening in a strategy vacuum.”

What works (and what doesn’t)

So if projects are already getting off the ground, what are feelings about where generative AI works best, and how?

“The best practises are undoubtedly cross-functional collaboration, ‘try before you buy,’ and learn from what you do,” says Marc O’Brien, CIO at radiology healthcare service provider Medica Group. “In my experience, the algorithms from reputable firms do what they say on the tin but what really matters is where you position in the workflow.”

Team Teach’s Ivell believes companies can gain a fast start by using tools being built into applications and suites.

“One of the key and immediate opportunities of generative AI is it’s already being built into some tools we already use, be that Power BI, Adobe or more industry-specific apps,” he says. “To take advantage of these needs some internal discovery or analysis of these new functions, understanding how we’d use them, and, in the first instance, training our staff how to exploit the new features. People tend to use tools in the way they always have, and adoption of new features can be slow, so we need to accelerate that.”

GigaOm’s Collins is an advocate of the always popular “start small” school of thought.

“Governance has to come first, given the answers offered by generative AI solutions come with risks and caveats,” he says. “From experience, text answers can be wrong, misleading, or incomplete, and code answers can be buggy or faulty. Starting small has to be the way forward, given that success with the tooling, at least currently, is often down to the ability to create well-formed questions.”

Ward-Dutton and IDC agree and add five other points of guidance: focusing on value and functionality, finding specific use cases, understanding limitations, considering the impact on work and jobs, and managing risks such as security, confidentiality, privacy and quality.

Obstacles and obstructions

Safety, bias, accuracy, and hallucination continue to be recurring issues.

Jon Cosson, head of IT and CISO at wealth management firm JM Finn, recalls asking ChatGPT for his own biography. The system listed only about 70% of his CV, and simply invented a period at a well-known bank.

“We need to realize where it can be enormously powerful and where it assists us, but be careful we retain human oversight,” he says. “It’s made my life easier because it allows me to write documents and make them richer, but if you rely on this beast it can bite you. We’re using it selectively in tests to see its power, but it’s heavily monitored and we won’t deploy anything if it causes any adverse decision making.”

Medica’s O’Brien issues a caution as well.

“Within healthcare the regulatory environment and the commercial frameworks are years behind the technology,” he says. “This makes it almost impossible to monetize, and, therefore, fund the implementation and usage of the algorithms. This is true across both public and independent sectors. That said, I believe once these barriers are overcome, benefits-led implementation will be swift.”

Coby adds that the immature regulatory and legal structures around using generative AI and large language models (LLM) need to be carefully considered, as does the tendency of current programs to hallucinate. “This is why, at this stage, it’s essential that any use is checked by someone with expert knowledge. Moving from PoCs to mainstream implementation will need to be carefully controlled.”

Ivell adds that generative AI could create unwelcome competitive dynamics.

“As part of our preparation of a generative AI strategy, it’s important to understand where this technology could enable competition or startups to use it to attack our market share with new tools producing faster-to-market and lower-cost products or services,” he says. “So there’s a lot to keep aware of—not just how we may exploit it but also keeping an eye on how it’s starting to be used as a threat.”

And in terms of intellectual property risks, IDC’s Ward-Dutton says oganizations’ own IP can leak into the public domain if they aren’t careful when using public generative AI services. “Some system providers are facing lawsuits because they trained their systems on data and content without getting permission from the original creators,” he says, adding that costs can also be prohibitive because the core technology powering today’s generative AI systems is very expensive to train.

Searching for the sweet spots

There are varying opinions where generative AI will make itself most felt. Collins nominates research and design: “It’s perfectly reasonable the challenges of creating a functional website from scratch should go away, as well as other areas that were already ripe for automation.”

O’Brien adds it’s anything that produces content for consumption by humans, where regulation is light and pricing can fund the algorithm.

IDC’s Ward-Dutton says the analyst’s customer panel points to three main clusters: improving customer and employee experiences; bolstering knowledge management; and accelerating software delivery. In time, he predicts, they’ll be joined by enterprise communication (including contact centres); collaboration and knowledge-sharing; content management; and design, research and creative activities.

Despite being too early to say, Coby believes initial successes will be in enabling humans to be much more productive by using generative AI to produce first drafts and then use them as foundations. “This is likely to be in multiple areas and will require new skills in asking the right queries,” he says.

Ivell concurs regarding areas of content, code generation, and customer support, but says he’s most excited by research opportunities.

“AI can analyze large volumes of data in textual form to create new forms, summaries, and analyses of the data sets,” he says. “It can also provide analysis of large data sets to produce enterprise-level insight previously unavailable such as understanding patterns in behavior and creating insight we can use to build new products.”

JM Finn’s Cosson, an enthusiastic blogger, says text and graphical content using tools such as Midjourney are obvious near-term opportunities.

“It’s already powerful in blog sites and a lot of people will use it as a framework,” he says. “You don’t want to lose the human creative element but you can apply human oversight elements and deliver some outstanding pieces. Where you see downsides are in marketing types and copywriters losing their jobs, but there will be new jobs created.”

A Trojan horse?

Some watchers believe that generative AI can be the trailblazer for wider application of AI and ML. IDC’s Ward-Dutton is particularly enthusiastic.

“In just a few months, generative AI has simultaneously captured the attention, imagination, and trepidation of tech and business leaders across the world,” he says. “We believe generative AI is a trigger technology that will usher in a new era of computing—the Era of AI Everywhere, which will completely change our relationship with data and how we extract value from both structured and unstructured data. The rapid adoption of generative AI moves AI from an emerging software segment in the stack to a lynch-pin technology at the center of a platform transition.”

But CIOs are vocal about the importance of robots working in tandem with people.

“AI works best when it works together with humans,” says Cosson. “The human brain is still worth something. Empathy and humanity are important and we need to work out how AI complements and fuses them together.”