Anthropic proposes bill to block AI model releases over catastrophic risk

The draft would let independent evaluators trigger a court review before a model is released if it appears to pose lethal or major financial harm.

Anthropic has circulated a draft AI safety bill that would create a formal process for stopping the release of a model when evaluators judge it could cause catastrophic harm. Under the proposal, the trigger is a foreseeable, imminent risk that could kill more than 50 people or cause over $1 billion in property damage, including risks tied to a biological weapon, a model escaping its developer’s controls, or a cyberattack. The bill would rely on independent evaluators registered with the U.S. Commerce Department, and if they flag danger, they would have 72 hours to notify the department. Commerce would then need to ask a D.C. judge to halt the release, and federal agencies would not be allowed to do that on their own. Anthropic told Politico the draft matches its Advanced AI Framework and said it has been discussing it with federal and state lawmakers since June.