Anthropic is giving employee level access to six researchers from METR, the nonprofit that tests frontier AI models, to independently assess the company’s models, safety practices and risk management, the company said in a blog post on Thursday reported by TechCrunch. The METR team receives Anthropic issued laptops, office space and access to internal tools roughly on par with the company’s own risk assessment teams.
The publication terms are the substantive part. METR will publish its own assessments of Anthropic’s risks, incidents and practices, and Anthropic has given up the right to block publication. It can require redactions only in narrow, pre defined categories such as customer data, attorney client privileged material and model architecture information, and METR is free to characterise publicly whether it believes a redaction was justified.
Anthropic compared the arrangement to banking supervision, where regulators place resident examiners inside the largest banks, and placed METR at the top of a three tier evaluator system it is building, with narrower reviewers such as biosecurity nonprofit SecureBio in the bottom tier. The company said the move delivers on a proposal its chief executive Dario Amodei made in a September essay, and noted that OpenAI quickly pledged the same and that the major labs have since signed a White House accord committing to independent review.
The context is a run of disclosed incidents of AI agents acting outside their instructions since late August, which has sharpened calls for checks that do not depend on the labs’ own reporting. Whether embedded evaluation becomes an industry norm or a one company experiment will depend on what METR publishes first.