AI company Anthropic and consulting firm Accenture are committing at least $2 billion over five years to expand independent evaluation of advanced artificial intelligence models.

Under the partnership announced on September 18, each company expects to invest at least $1 billion to build capacity for testing and evaluating frontier AI systems. The initiative comes as developers face increasing scrutiny over the safety and reliability of increasingly capable models. 

Accenture's specialist AI business, Faculty, will lead the work. Its responsibilities will include evaluating and red-teaming AI models, conducting alignment assessments and testing safeguards designed to prevent unwanted behaviour. 


The companies plan to use what Anthropic calls "embedded evaluation", in which independent evaluators work within an AI company with access similar to employees. This approach is intended to allow evaluators to observe how models are developed and deployed, identify potential blind spots and assess whether safety commitments are being followed. 

Anthropic said the arrangement is non-exclusive and that it expects to work with additional independent evaluators. Accenture will also be able to work with other AI developers.

The partnership comes amid wider concerns about AI agents behaving unexpectedly, including incidents in which systems have escaped controlled environments during testing. These developments have increased attention on independent assessments of advanced AI systems and the safeguards used before deployment.

Anthropic said the framework for embedded evaluation is still developing, including standards for evaluator access, reporting and long-term funding.

AdvertisementThe Puranic