Anthropic Details Its Framework for Assessing AI Harms

Anthropic outlines harm-assessment framework

Anthropic said it is sharing its evolving approach to assessing and mitigating a range of potential harms from its AI systems, from catastrophic risks such as biological threats to concerns including child safety, disinformation and fraud.

The company said the framework complements its Responsible Scaling Policy, which it said focuses specifically on catastrophic risks, by taking a broader look at other kinds of impacts. Anthropic described the approach as still evolving and said it is sharing current thinking that will continue to develop.

Five dimensions of harm

Anthropic said it evaluates potential impacts across five baseline dimensions:

• Physical impacts: effects on bodily health and well-being
• Psychological impacts: effects on mental health and cognitive functioning
• Economic impacts: financial consequences and property considerations
• Societal impacts: effects on communities, institutions and shared systems
• Individual autonomy impacts: effects on personal decision-making and freedoms

For each dimension, the company said it considers factors including likelihood, scale, affected populations, duration, causality, the technology's contribution to the harm, and the feasibility of mitigation.

Anthropic said it manages risks identified through the framework using a combination of measures: a Usage Policy, evaluations such as red teaming and adversarial testing conducted before and after launch, detection techniques aimed at identifying misuse, and enforcement actions ranging from prompt modifications to account blocking.

Examples cited

Anthropic pointed to two examples of how it applied the framework.

For its computer use capability, which lets models interact with computer interfaces, the company said it examined risks tied to financial software and banking platforms, where it said unauthorized automation could facilitate fraud or manipulation, as well as communication tools that it said could be used for targeted influence operations or phishing. Anthropic said this analysis led it to set more stringent enforcement thresholds and to use hierarchical summarization, an approach it said lets the company detect harms while maintaining its privacy standards.

The company also described work on what it called model response boundaries, covering how models handle requests that fall between clearly harmless and clearly harmful. Anthropic said models trained to be more helpful can lean toward harmful behavior, while models that over-index on harmlessness can refuse to share information even when a request is harmless. With Claude 3.7 Sonnet, the company said it evaluated different types of requests along this spectrum and changed how the model handles ambiguous prompts, favoring safe, helpful responses over refusal. Anthropic said this produced a 45% reduction in unnecessary refusals while maintaining what it described as strong safeguards against harmful content. The company said this kind of analysis also informs where it focuses safety evaluations for groups it identified as potentially at heightened risk, including children, marginalized communities and individuals in crisis.

Still evolving

Anthropic said its approach to understanding and addressing harms is one input into its broader safety strategy and said it expects new challenges to emerge as AI systems become more capable. The company said it invited researchers, policy experts and industry partners to collaborate on the work, pointing to the email address usersafety@anthropic.com for contact.

Source: Anthropic, "Our approach to understanding and addressing AI harms," published April 21, 2025.