GAO report says DHS, other agencies need to up their game in AI risk assessment

A new report from the US Government Accountability Office (GAO) appears to indicate that no US federal agency reporting into the Department of Homeland Security (DHS) knows the full extent or probability of harm that AI can do the nation’s critical infrastructure.

In the report, released earlier this week, it concluded the DHS needs to improve its risk assessment guidance, noting there was one area in which every agency failed: “None of the assessments fully evaluated the level of risk in that they did not include a measurement that reflected both the magnitude of harm (level of impact) and the probability of an event occurring (likelihood of occurrence). Further, no agencies fully mapped mitigation strategies to risks, because the level of risk was not evaluated.”

AI, report authors noted, has the “potential to introduce improvements and rapidly change many areas. However, deploying AI may make critical infrastructure systems that support the nation’s essential functions, such as supplying water, generating electricity, and producing food, more vulnerable.”

Nobody knows the probability of harm

The GAO said it is “recommending that DHS act quickly to update its guidance and template for AI risk assessments to address the remaining gaps identified in this report.” DHS, in turn, it said, “agreed with our recommendation and stated it plans to provide agencies with additional guidance that addresses gaps in the report including identifying potential risks and evaluating the level of risk.”

Peter Rutten, research vice president at IDC, who specializes in performance intensive computing, said Friday that his take is, “indeed, no DHS agency knows the full extent or probability of harm that AI can do to the US critical infrastructure. I’d argue that, today, no entity knows the full extent or probability of harm that AI can do in general — whether it is an enterprise, government, academia, you name it.”

AI, he said, “is being pushed out to businesses and consumers by organizations that profit from doing so, and assessing and addressing the potential harm it may cause has until recently been an afterthought. We are now seeing more focus on these potential negative effects, but efforts to contain them, let alone prevent them, will always be far behind the steamroller of new innovations in the AI realm.”

Thomas Randall, research lead at Info-Tech Research Group, said, “it is interesting that the DHS had no assessments that evaluated the level of risk for AI use and implementation, but had largely identified mitigation strategies. What this may mean is the DHS is taking a precautionary approach in the time it was given to complete this assessment.”

Some risks, he said, “may be identified as significant enough to warrant mitigation regardless of precise quantification of that risk. Moreover, some broad mitigation strategies are valuable to implement regardless of specific risk (such as ensuring explainability or having regular audits).”

According to Randall, “given the lead agencies only had 90 days to complete their assessments, choosing to document broader risk mitigation strategies achieves broader value than individually evaluating the level of each risk. This is a task that should come next, though, now that use cases, risks, and broader mitigation strategies have been identified.”

The report noted that federal agencies with a lead role in protecting the nation’s critical infrastructure sectors, referred to as sector risk management agencies (SRMAs), were required to develop and submit initial risk assessments for each of the critical infrastructure sectors to DHS by January 2024, in coordination with DHS’s Cybersecurity and Infrastructure Security Agency (CISA).

However, it said, “although the agencies submitted the sector risk assessments to DHS as required, none fully addressed the six activities that establish a foundation for effective risk assessment and mitigation of potential artificial intelligence (AI) risks. For example, while all assessments identified AI use cases, such as monitoring and enhancing digital and physical surveillance, most did not fully identify potential risks, including the likelihood of a risk occurring.”

Rutten didn’t find this unreasonable. He noted, “it’s entirely fair that the agencies were unable to assess the extent or likelihood of harm. There are thousands of algorithms in circulation — some proprietary, some open source — each with its own development history, data used to train, accuracy rates, and hallucination probability, not to mention vulnerabilities.”

Not until preventing harm at the foundation of algorithm development becomes the norm (and mandatory), he said, will it be possible to determine how safe these algorithms are. “Investigating, testing, and assessing them all is impossible, not in the least because an algorithm may iterate harmlessly millions of times, and then suddenly make one crucial mistake,” he said. “In other words, the horse is out of the barn, and I don’t see how we are going to catch up with it for the foreseeable future.”