Senators are drafting an AI duty of care that could require developers of the most advanced systems to prevent known catastrophic risks before releasing their models. The bipartisan talks would place legal responsibility on frontier laboratories rather than relying only on voluntary safety promises.

 

The proposal remains under negotiation, and no final bill text or vote schedule has been announced. Still, the discussions mark a significant shift in Washington: lawmakers are considering both enforceable developer obligations and a federal mechanism capable of stopping a model release judged unsafe.

 

Senate AI Duty of Care Would Target Frontier Developers

Reuters reported on September 11 that Senate negotiators are considering a duty-of-care standard for companies building advanced AI. The measure would focus on catastrophic hazards, including the possibility that a powerful model could materially assist the creation of nuclear or biological weapons.

 

The developing framework is aimed primarily at frontier-model companies such as OpenAI, Anthropic and Google. That makes capability and risk, rather than the broad use of ordinary software, the center of the proposal described by people familiar with the talks.

 

The negotiations reportedly cover four central mechanisms:

  • A legal duty to mitigate known catastrophic risks.
  • Federal authority to block an unsafe model release.
  • A court process for companies to challenge a block.
  • Potential testing support from U.S. national laboratories.

 

Those elements are not yet settled. Details could change before legislation is introduced, and the Senate’s compressed calendar ahead of the November elections leaves uncertain whether negotiators can convert the framework into a bill and move it through committee.

 

Federal Model-Release Blocks Would Create a New Safety Gate

The most consequential idea is authority for the federal government to prevent the release of an AI model deemed unsafe. Existing U.S. oversight is fragmented across agencies, state laws, procurement rules and voluntary company commitments; a release gate would be a more direct intervention in frontier development.

 

The reported court-review provision would give a company a path to contest a government decision. That safeguard could become central to the legislation because a release block would affect intellectual property, commercial timing and competition, while regulators would need a defensible technical basis for finding that a model presents catastrophic danger.

 

National-laboratory testing could help provide that basis. U.S. labs have secure facilities and experience evaluating technologies tied to nuclear, biological and national-security risks, although lawmakers would still need to define testing standards, access rules, confidentiality protections and what level of capability triggers federal scrutiny.

 

Further Reading

 

State-Law Preemption Is a Major Point of Tension

Negotiators are also discussing whether federal rules should preempt state laws addressing catastrophic AI risks. Technology companies often favor one national framework over a patchwork of state requirements, while state officials and safety advocates may resist losing protections when Congress has not yet enacted a comprehensive federal regime.

 

The issue is especially sensitive after California expanded its AI and online-safety rules. A narrow preemption clause limited to the same catastrophic-risk field would operate differently from a broad ban on state AI regulation, so the final wording would determine how much authority states retain.

 

Senate Majority Leader John Thune and Commerce Committee leaders Ted Cruz, Amy Klobuchar and Maria Cantwell are among the lawmakers involved in the bipartisan discussions, according to Reuters. Committee jurisdiction matters because any agreement would likely need to survive both technical policy disputes and the ordinary legislative process.

 

Frontier Labs Face Pressure for Binding National Rules

The negotiations arrive as leading developers publicly acknowledge that voluntary commitments may be insufficient for the most capable systems. OpenAI has advocated mandatory, capability-based national safety requirements, while Anthropic’s threat reports have documented attempts to misuse Claude for cyber operations, weapons-related research and espionage.

 

A statutory duty of care would differ from a checklist. It could require a developer to show that its safeguards reasonably address known severe risks as capabilities evolve, potentially making evaluations, cybersecurity controls, incident reporting and release decisions evidence in a future enforcement action.

 

That flexibility also creates a hard drafting problem. If “catastrophic risk” or “known risk” is vague, firms may not know what compliance requires; if Congress defines the terms too narrowly, the law may miss new capabilities and attack paths that emerge after enactment.

 

The next concrete signals will be public bill text, a Commerce Committee hearing or markup, and a defined role for the federal agency or laboratories responsible for evaluations. Until then, the proposal is best understood as an important bipartisan negotiation—not an enacted release-approval system or a finalized national AI law.