AI Giants Face Backlash Over Control of Independent Safety Reviews
Anthropic and OpenAI face pressure from experts to allow transparent AI safety evaluations, as concerns grow over corporate control of research findings.
POLICY WIRE — New York, USA — Leading artificial intelligence developers Anthropic and OpenAI are facing intense scrutiny regarding their influence over independent safety evaluations. While both companies have expressed a desire to integrate external reviewers into their labs, critics argue that current contractual constraints threaten the transparency of these assessments.
According to individuals familiar with the matter, both firms have sought significant authority over what outside evaluators are permitted to publish. These companies are reportedly concerned that unrestricted disclosures could inadvertently expose sensitive intellectual property and trade secrets.
A review of internal documentation reveals that specific contract language currently prohibits evaluators from releasing information without prior company consent. This dynamic has sparked a debate among researchers who warn that without standardized, independent oversight, the labs building the most powerful models retain too much power to dictate how their systems are scrutinized.
The push for greater accountability has gained momentum, with a coalition of more than 100 prominent AI experts signing a public letter demanding increased independence and transparency. This group, which includes Canada’s renowned godfather of AI, is urging foundation model labs to move away from restrictive practices.
In response to these concerns, the developers of the Claude AI models stated that they have clarified to their current evaluation partners that the reviewers maintain control over their own methodologies, criteria, and final conclusions, as well as the decision on what to publish.
Reporting by Policy-Wire (PW)
