AI is paying off, but governance is lagging behind

Enterprises are facing two simultaneous challenges with AI: The risks associated with it are evolving faster than governance frameworks, while the business benefits are often difficult to measure.

This is one of the key findings of The Value of AI, a study commissioned by SAP from Oxford Economics. Now in its second year, the study surveyed 2,600 executives from 13 countries worldwide.

High expectations, limited preparation

On average, the enterprises surveyed plan to spend around $28 million on AI (up from $26.7 million last year), and expect a 21% ROI (from 16% last year). Expectations for AI agents are particularly high, with ROI expected to reach 17% this year, up from 10% last year. Furthermore, 83% of respondents worldwide said agentic AI has the potential to fundamentally transform their organization. On the other hand, only 3% of respondents said their enterprises were fully prepared for the deployment of AI agents.

There are gaps, particularly when it comes to governance:

• Only 12% of respondents said their skills or processes were able to govern AI effectively,
• 38% do not have human-in-the-loop processes in place for oversight of AI agents, and
• only 63% have established permissions and access controls for agents.

Other concerns include weaknesses in the organization of AI deployment, poor data quality, insufficient employee training, and the widespread use of shadow AI.

Governance is the bigger challenge

In an interview, Sean Kask, Chief AI Strategy Officer at SAP, commented on the study’s key findings.

Mr. Kask, in the study’s foreword, you write that companies are currently facing two challenges simultaneously: The risks associated with AI are evolving faster than governance, while the business benefits are often difficult to measure. Which of these poses the greater problem for companies?

Sean Kask: Measuring the business value of IT investments has never been easy. The same applies to AI. That’s why I currently consider the governance issue to be the greater challenge. While traditional governance principles and best practices for secure software development remain important even in the age of large language models and agent-based AI, entirely new risks are emerging at the same time.

For example, as soon as companies roll out AI on a broad scale, they suddenly discover hundreds or even thousands of so-called shadow agents that employees are using without central oversight. Or they find that a significant portion of the workforce is copying content into private ChatGPT accounts. Such risks often only become apparent once AI is already being used productively.

According to your study, German companies invest an average of nearly $40 million in AI, more than companies in all other countries surveyed. Why is that?

Kask: I was less surprised by the amount of investment than by the fact that, overall, the level of investment and the return on investment achieved have developed very similarly across the various countries. There’s no clear answer as to why Germany invests more. In part, it’s likely simply because costs here are higher than in India, for example.

However, we’re also seeing a high level of AI adoption among German companies. SAP has a dashboard that allows us to track how our customers are using AI features. Germany is among the countries with particularly high usage. Added to this are the strong industrial base and the political impetus from Europe, which are driving the use of AI. Accordingly, companies there are making targeted investments in building the necessary expertise.

According to the study, 47% of German companies are satisfied with the return on investment from their AI investments. At the same time, 77% say they are still far from realizing AI’s full potential. Isn’t that a contradiction?

Kask: No, we see this pattern worldwide. Companies initially invest in a few AI use cases and realize: This works; we’re creating added value. Accordingly, they’re satisfied with their investment.

But this is precisely what leads them to identify further use cases. They explore AI agents and want to utilize them as well. However, it is exactly at this point that many encounter new challenges in implementation and scaling.

The study therefore primarily highlights a learning curve: The more experience companies gain with AI, the greater their awareness of its previously untapped potential becomes.

According to the study, only 33% of companies surveyed have KPIs at the executive board level that are directly linked to the implementation of AI. In your view, which metrics should supervisory boards and CEOs definitely be tracking?

Kask: For us, a key indicator is employee enablement. How many employees have already successfully completed training or upskilling programs related to AI? Without the appropriate skills, AI adoption will fall short of its potential.

Transparency is equally important. Companies should know which AI agents are actually in use within their landscape. SAP offers the SAP AI Agent Hub for this purpose, which automatically discovers and inventories agents from SAP and third-party environments. Customers have already been able to identify thousands of agents this way, which highlights the need for centralized governance and transparency.

In addition, companies should have a complete overview of all AI use cases. A robust business case should be in place for each use case. We often see two extremes: Either the executive board is under pressure to implement AI as quickly as possible and allocates a lump-sum budget for this purpose. Or management initially takes a wait-and-see approach. This leads to independent pilot projects springing up throughout the company, with individual departments procuring their own tools and entering into their own contracts.

At SAP, we therefore follow a clearly structured selection process. Each idea first undergoes an assessment of its expected business value. We then examine technical feasibility, data availability, and ethical and governance aspects. From management’s perspective, it is crucial to maintain transparency regarding all ongoing AI projects at all times and to consistently prioritize them based on their business value.

Agents, too, need a ‘hire-to-retire’ lifecycle

Even with the introduction of dozens or even hundreds of AI agents, governance becomes increasingly complex. What capabilities do enterprise platforms need to manage AI agents securely and in a controlled manner at scale?

Kask: We make a conscious effort not to anthropomorphize AI too much. Nevertheless, the analogy is helpful: Agents require a complete hire-to-retire lifecycle. This begins with the detection and registration of an agent. It is then integrated into the enterprise environment, granted the necessary permissions, and given access to the data sources it needs to perform its tasks.

Observability is just as important. Companies must be able to track what an agent is actually doing in the system at all times. In addition, they should track key performance indicators: Is the agent achieving the desired results? How efficiently is it working? How many tokens does it consume? How many processing steps does it require for a task?

Ultimately, this involves several key components: a complete inventory of all agents, appropriate governance, risk, and compliance (GRC) mechanisms, transparency regarding agent behavior, and continuous monitoring. This is the only way to ensure that AI agents consistently operate within defined parameters and deliver the desired business value.

In your estimation, which business processes will companies actually delegate entirely to AI agents over the next two to three years?

Kask: Currently, such agents work particularly well in clearly defined use cases. SAP will release more than 50 (currently 34) specialized AI agents.

One example is periodic financial reporting. In this context, journal entries must be made based on numerous rules stored in documents, emails, or previous transactions. The agent analyzes these various sources of information, derives a recommendation from them, and suggests the appropriate journal entry to the user.

Based on what we’ve heard from customer projects, employees at medium-sized companies currently spend about twelve hours per month on these tasks. With the help of an AI agent, this effort can be reduced to two to three hours.

Another area of application is production planning. If delivery dates change or new orders come in at short notice, the entire production plan must be adjusted. It is precisely these kinds of complex optimization tasks that are ideally suited for AI agents.

In principle, there are virtually no limits to the narrowly defined business processes in which agents can be deployed. However, they will not operate completely autonomously at first.

Trust in AI begins with a stable foundation

Many companies still struggle to trust AI agents. After all, large language models operate probabilistically and can produce false information. This is particularly problematic in financial processes. How do you build trust?

Kask: Trust begins with a stable foundation. ERP systems remain the reliable system of record. They operate deterministically, contain the business logic, and hold the relevant company data. AI agents build upon this foundation. They do not replace it.

Equally important is the human-in-the-loop principle. Employees must be able to understand what the agent is doing, verify its results, and intervene if necessary. That’s why employee training also plays a crucial role. They must understand how generative AI works and where its limitations lie.

Of course, language models can hallucinate. At the same time, we must not forget that humans are not infallible either. The key lies in the collaboration between humans and AI. This allows us to improve both the efficiency and the quality of many business processes.

Another important component is transparency. Our global AI ethics policy, for example, stipulates that users must always be able to recognize when AI is involved. In Joule, it’s possible to trace which data sources the agent used and which steps it went through in reaching its decision. This traceability is an essential prerequisite for trust.

What distinguishes an SAP agent from a general AI agent that merely accesses an ERP system?

Kask: The key difference is that Joule and the SAP agents are directly embedded in the ERP system. There, for example, we’ve built a knowledge graph that describes the semantic relationships between all tables, business objects, and data fields.

To put this into perspective: The SAP S/4HANA Knowledge Graph is based on approximately 452,000 ABAP tables, 7.3 million data fields, and thousands of analytical views. The semantic relationships between these artifacts are modeled in the Knowledge Graph and made available for AI applications.

For example, if a user wants to view all open purchase orders, the agent does not first have to laboriously search for the relevant information. It immediately knows which tables and objects are relevant and also understands the relationships between a purchase order, a purchase requisition, the responsible approvers, and other business objects. As a result, the agent not only works much more precisely but also requires significantly fewer tokens because it can greatly narrow down the search space.

If, instead, one attempts to simply overlay AI onto an existing system or extract data from a relational ERP system, many of these relationships are lost. In a sense, this destroys the semantic context that is crucial for precise answers.

That is why we view the ERP system as an enormous strategic advantage. It has been the system of record for decades and contains roughly 50 years of codified business and process knowledge. This knowledge forms the foundation for what we call the autonomous enterprise. The agents build upon this knowledge and continue to develop it.

In the future, SAP agents will also communicate bidirectionally with agents from other providers via standards such as Agent-to-Agent (A2A).

According to your study, AI currently creates the greatest added value in decision-making, customer interaction, and gaining new insights, rather than in traditional productivity gains. Will this change the way companies justify AI investments in the future?

Kask: In our study, productivity was simply rated slightly lower than, for example, gaining new insights. In the long term, however, productivity remains the ultimate goal. Europe, in particular, has been suffering from comparatively weak productivity growth for years.

At SAP, we therefore first evaluate every new AI feature based on its specific business value. For all agents and AI features that we include in our AI Feature Catalog, we first conduct a value analysis. We ask: What benefit does the feature offer the user? Does it contribute to higher revenue? Does it increase productivity? Only then is it developed further.

At the moment, the greatest added value often still lies in consolidating information from structured and unstructured data sources and making it accessible via natural language. The next step, however, is to translate these insights directly into more efficient business processes. That is precisely where the greatest productivity gains will be realized in the future.

If you could give CIOs just one or two pieces of advice for the transition from generative AI to AI agents, what would they be?

Kask: In my view, the biggest mistake would be to try to transform the entire company all at once or to attempt to perfectly prepare all the data right from the start.

Instead, you should consider what kind of agent can create significant added value, and then implement it. Of course, this agent needs access to consistent and context-rich enterprise data. That’s exactly what we’re working on at SAP with technologies like the knowledge graph, which maps the semantic relationships within enterprise data.

In addition, with data products and the SAP Business Data Cloud, we provide tools that make data from various sources usable for AI agents. Thanks to zero-copy and data fabric approaches, information from legacy systems, Snowflake, or ERP systems can be consolidated without first having to extensively replicate the data. For a procurement agent, this makes it possible to provide exactly the relevant data for the specific use case.

The key point is this: Companies do not have to wait until they have fully migrated to the cloud or consolidated their entire data landscape. With the technologies available today, data can already be made usable for specific AI agents, managed in a controlled manner, and used to quickly generate initial business value. On the other hand, those who wait for the perfect starting point run the risk of falling behind.

This article is adapted from one first published by Computerwoche.

---