Independent AI safety evaluators are moving from the industry's margins into central roles as Anthropic and OpenAI face public pressure to prove they can safeguard advanced models while scaling aggressively.
The shift gained momentum after Anthropic CEO Dario Amodei pledged last month to embed third-party evaluators inside his company. OpenAI CEO Sam Altman endorsed the approach. President Donald Trump and most major U.S. tech companies backed the idea, and Trump recently encouraged AI companies to "partner with an independent external auditor or evaluator" as part of a voluntary accord announced in late September.
Small nonprofit and for-profit groups including METR, Apollo Research, Transluce, and Vals AI now occupy the center of this emerging ecosystem. These organizations assess AI model capabilities, flag risks, and conduct safety assessments. METR announced in August that it had raised commitments of around $71 million over six months, up from total 2024 contributions of $13.6 million according to IRS filings. Vals AI, a for-profit startup, grew from eight employees to roughly 30 this year and announced a $40 million funding round in August.
Yet critical gaps remain unresolved. Suresh Venkatasubramanian, a computer science professor at Brown University, told CNBC that funding models pose an immediate challenge. "Who is paying for these companies to do their work? How are they going to support them? You need an ecosystem, you need a viable business model for this," he said.
Friction has already emerged. OpenAI fired three employees last week for "violating our policies on accessing and handling sensitive company information." Two of those employees, Mikita Balesni and Tomek Korbak, said they believe they were dismissed because of how they communicated with third-party evaluators. Balesni wrote on X that colleagues feared speaking up and worried their personal phones would be searched. OpenAI disputed that account and said it is "actively finalizing contracts with third-party safety assessors and will announce details in the coming weeks."
Kevin Werbach, faculty director of the Wharton Accountable AI Lab, said the ecosystem is "not robust enough right now." METR employs fewer than 50 full-time staffers. The power imbalance between small evaluators and labs that have raised tens of billions of dollars raises questions about independence. Venkatasubramanian said true third-party evaluation requires financial independence, adding "it's not just a matter of not getting paid, it's a matter of" maintaining operational autonomy.