OpenAI’s new priorities for third-party assessments are a fine start, but they lack teeth

OpenAI published a detailed list of priorities and principles on Tuesday designed to govern its third-party model assessors, a list that industry observers agreed was a good one. But they also stressed that it lacked any enforceable controls to truly maintain safety.

If the goal is to encourage enterprise CIOs to trust OpenAI more, it won’t help, they said, but most doubted that this was OpenAI’s objective. Its more probable aim is to use the list to work out an arrangement with competitors and regulators so that enforcement is made more palatable, under the theory that it is open to stricter enforcement, but only if all of its key rivals are locked into identical restrictions.

The OpenAI post said, “OpenAI is committed to supporting independent assessments with deep levels of access across training, evaluation and deployment. That access should enable assessors to challenge our assumptions, identify risks we may have missed, and reach their own conclusions about the effectiveness of our safeguards.”

It added: “Third party assessments are most useful when they address specific, consequential questions: Does the evidence support a lab’s safety case and safety claims? Do evaluations adequately test the risks they are intended to measure? Do safeguards work under realistic conditions?”

Nothing about enforcement

Pieter Arntz, a malware intelligence researcher at Malwarebytes, said that giving third parties rules for engagement is certainly a good thing, but the implication behind such rules is that they are somehow enforceable. And the document says nothing about that enforcement process.

“It sets some useful expectations for independence and rigor, but it does not itself compel OpenAI to submit to a particular scope, publish adverse findings, or change deployment decisions,” he pointed out. “Its credibility will depend on the terms of individual assessments, and what outsiders are allowed to see. Its value therefore hinges on whether OpenAI accepts genuinely inconvenient scrutiny, and whether results, redactions, remediation, and deployment decisions can be independently checked.”

Valence Howden, advisory fellow at Info-Tech Research Group, agreed that the lack of enforceability makes the rules close to pointless. 

The document “doesn’t impose any requirements on OpenAI, and I would have been surprised if it did, because their motivations are not as clear-cut as they’re stating,” he said. “They also haven’t really identified what they will do based on the assessments, once they are conducted. OpenAI retains control over scope in that [OpenAI] must agree with it, limiting where assessors can go. They retain the ability to determine the appropriate level of access, which still allows them to filter what’s provided, impeding the gathering of direct evidence in those cases they define as material or classified. It also allows them to control much of the reporting and redact portions.”

Howden said the net impact is that “it’s good for determining what should be assessed, but considerably weaker on true clarity and approaches for how this is done.”

But Jason Andersen, principal analyst at Moor Insights & Strategy, offered a different perspective, although he reached the same conclusions as Howden and Arntz.

“It’s not an act of good faith to not empower these evaluators,” he said, but at the same time, CIOs need to understand the split personality of OpenAI today.

“It’s a classic problem with OpenAI because they are a split brain company, with the commercial side and the original mission side,” he pointed out. “Those two teams have not aligned very well over the past two years. It looks like it started with the original mission team, but then commercial interests took over that document and turned it into legalese.”

‘A principle instead of an excuse’

Frank Dickson, principal analyst at Dickson Research, concluded from the post that OpenAI “seems proactive in highlighting the threats posed by AI, but hesitant to accept accountability or take real action. This is a code of conduct for the assessors, not for OpenAI. The gap between the two is the whole story.”

He delved into the details of the document and was not happy. He pointed out, for example, that third-party assessor access was “granted within the bounds of legal, security, and IP constraints. That is a company deciding the scope of its own scrutiny.”

He also noted that the document specified that “publication happens after labs get a reasonable period to remediate issues. That is the same grace-period logic that just got Google criticized for sitting on the Gemini hacking disclosure for seven weeks, now written down as a principle instead of an excuse.”

Just the beginning

However, Samantha Gloede, global head of risk services at KPMG, said that she sees the document as the beginning of the process, and it should be evaluated accordingly.

“I believe the conversation will increasingly shift from assessment to accountability,” Gloede said. “Independent testing is important, but long-term trust depends on how organizations respond when material risks are identified. Stakeholders will want confidence not only that issues can be found, but that corrective actions are taken, independently validated and reflected in governance and deployment decisions. The next evolution of AI assurance will be demonstrating that accountability with the same rigor used to assess risk in the first place.”

Edna Conway, executive advisor at consulting firm Acceligence, agreed, noting that the substance of this effort will only become clear in the next phase. 

“The next step is making the lab accountable to the assessment process itself. Who determines when an assessment is required? ” she asked. “How much access is sufficient? What happens when the assessor and lab disagree about scope? Who sees a critical finding? Embracing independence is essential, as third-party assessments are genuinely valuable only when the assessor can ask uncomfortable questions, get sufficient access to answer them, follow evidence outside the original hypothesis when necessary, and report conclusions without commercial or organizational pressure shaping the outcome.”