Anthropic Grants External Watchdog "Employee-Like" Access for AI Risk Assessment
Anthropic has given the nonprofit METR extensive access to its internal systems to independently evaluate AI risks, marking a key step in industry safety pledges.
Highlights
- AI developer Anthropic has provided "employee-like" access to METR, an independent nonprofit, to evaluate the risks associated with its AI models.
- A team from METR will operate within Anthropic's offices, utilizing company hardware and internal systems for their assessments.
- This initiative responds to growing concerns about the potential dangers advanced AI systems could pose to cybersecurity, the economy, and humanity.
- METR is authorized to publish its findings without requiring Anthropic's consent, though the company can request redactions for specific sensitive data categories.
- The arrangement underscores Anthropic's commitment to transparency and accountability in its AI safety protocols.
Details
This move by Anthropic follows a commitment made by its CEO, Dario Amodei, who, alongside OpenAI's Sam Altman and SpaceX's Elon Musk, advocated for independent auditors within leading AI labs. The recent announcement represents the initial implementation of these safety pledges. Anthropic stated that as AI models become more sophisticated, they necessitate more robust safeguards and thorough risk evaluations. The company aims for public accountability in its decisions while acknowledging that not all safety-related information can be disclosed publicly.
METR, which previously investigated a security incident involving OpenAI models at Hugging Face, has now integrated six of its personnel into Anthropic. These individuals have been granted company equipment, office space, and access to internal tools and systems comparable to what Anthropic's own risk assessment teams use. Anthropic is not financially compensating METR for this arrangement. METR operates independently, receiving its funding from private donations, philanthropic organizations, and governmental entities like the UK's AI Security Institute, rather than from AI companies or their personnel.
Why it matters
This development signifies a concrete action within the AI sector to address increasing safety concerns. The ability of external evaluators to report independently has the potential to enhance transparency and accountability among AI developers. Furthermore, such collaborations can contribute to better management of the emerging risks associated with AI technology and help build public trust. Anthropic is actively encouraging other prominent AI companies to adopt similar measures, aiming to establish a new standard for industry-wide safety and oversight.