Anthropic updates its risk governance framework

The launch this week by Anthropic of an update to its Responsible Scaling Policy (RSP), the risk governance framework it says it uses to “mitigate potential catastrophic risks from frontier AI systems,” is part of the company’s push to be perceived as an AI safety first provider compared to its competitors such as OpenAI, an industry analyst said Wednesday.

Thomas Randall, director of AI market research at Info-Tech Research Group said that while there will not be immediate business benefits that come from the changes, the firm’s founding was “grounded in two OpenAI executives leaving that company due to concerns about OpenAI’s safety commitment.”

In the executive summary of the updated RSP, Anthropic stated, “in September 2023, we released our Responsible Scaling Policy (RSP), a public commitment not to train or deploy models capable of causing catastrophic harm unless we have implemented safety and security measures that will keep risks below acceptable levels. We are now updating our RSP to account for the lessons we have learned over the last year. This updated policy reflects our view that risk governance in this rapidly evolving domain should be proportional, iterative, and exportable.”

The RSP, said Anthropic, is “based on the principle of proportional protection: safeguards that scale with potential risks. To do this, we use AI Safety Level Standards, graduated sets of safety and security measures that become more stringent as model capabilities increase. These begin at ASL-1 for models that have very basic capabilities (for example, chess-playing bots) and progress through ASL-2, ASL-3, and so on.”

In announcing the update on Tuesday, the company said that while at present its AI models operate under the ASL-2 standard, “which reflect current industry best practices,” the updated policy defines what it describes as “two key Capability Thresholds that would require upgraded safeguards.”

The two are:

• Autonomous AI Research and Development: Anthropic said, “If a model can independently conduct complex AI research tasks typically requiring human expertise — potentially significantly accelerating AI development in an unpredictable way — we require elevated security standards (potentially ASL-4 or higher standards) and additional safety assurances to avoid a situation where development outpaces our ability to address emerging risks.”
• Chemical, Biological, Radiological, and Nuclear (CBRN) weapons: According to the company, “If a model can meaningfully assist someone with a basic technical background in creating or deploying CBRN weapons, we require enhanced security and deployment safeguards (ASL-3 standards).”

Asked how the firm is actually going to achieve such goals when it comes to CBRN weapons, Randall said in an email that Anthropic’s updated RSP is designed to reflect the assertion that their current models would not be allowed to operate in certain contexts.

For instance, he said, “in the event of Anthropic’s models being requested for the creation of CBRN weapons (i.e., federal departments), Anthropic reinforces that their models would need to be at ASL-3 standards before enablement. Of course, much of the text around these contexts is vague, including what a ‘multi-layered approach’ looks like to prevent deployment risks.”

However, said Randall, “the RSP is not intended to be the legal outline of this policy; it is a public-facing summary highlighting the threshold at which Anthropic would consider allowing their models to be used in high-risk scenarios.”

As for autonomous AI research and development, he was asked how Anthropic decides at which point what they have created crosses over from being a competitive advantage for their R&D team into a clear and present danger to humanity. He replied, “as a general statement, no for-profit organization can be trusted to be self-organizing in the public interest.”

According to Randall, that is why “we have levels of regulations to keep organizations in check and to hold them accountable if regulations are broken. We cannot trust Anthropic to make their own calls correctly, though the RSP is a step in the right direction (it may even be more than most regulatory bodies have produced on the topic).”

The best we can do, he said, “is support our legal and political systems to keep up with rapid developments in AI and enforce rules that all AI-based companies must follow, setting the frameworks that all RSPs should fall within. The EU is certainly leading the charge on that front.”