New York's AI chatbot law bans unsafe companion features for minors and requires operators to verify that users are adults before offering them. Governor Kathy Hochul signed S.9051B/A.10379C on October 28, creating one of the country's most detailed state rules for consumer chatbots.

 

The law targets systems that imitate close relationships, encourage harmful conduct or use personal information to deepen emotional dependence. It takes effect 180 days after signing and applies when covered chatbot conduct occurs wholly or partly in New York.

 

The prohibited features cover six main areas:

  • Pretending to be human or to have human emotions
  • Claiming romantic, family, professional or authority relationships
  • Encouraging self-harm, eating disorders or substance misuse
  • Pressuring users toward secrecy or social isolation
  • Using older health or personal information to simulate companionship
  • Producing sexually explicit content or child sexual abuse material

 

New York AI Chatbot Law Draws a Bright Line

The statute defines an advanced chatbot as a generative AI system with a written or spoken natural-language interface that provides ongoing, adaptive responses. The restrictions reach both developers and operators that create or make those systems available to users in New York.

 

Its definition of unsafe companionship is unusually specific. A chatbot may not suggest that it is a real or fictional person, describe itself with human emotions, frame output as personal opinion, or prioritize flattery over a user's safety when the protected rules apply.

 

The bill also limits memory-based intimacy. It treats the use of health, wellbeing or personal information obtained more than 12 hours earlier, or during a previous session, as an unsafe feature when that information helps simulate companionship.

 

Other provisions prohibit prompts that encourage secrecy, isolation or avoidance of parents and professionals. Systems cannot promote suicide, self-harm, serious physical harm, disordered eating, unlawful drug use or alcohol abuse, and they cannot generate sexually explicit interactions for covered minors.

 

Age Checks Must Protect Privacy

Operators cannot offer the covered features unless they have determined that a user is not a minor. The law requires more than one way to verify age, including at least one option that does not rely solely on government-issued identification or that preserves anonymity from the operator.

 

Information gathered for that age determination cannot be reused for another purpose. It must be deleted immediately after the verification attempt unless retaining it is necessary to comply with another state or federal requirement.

 

Those conditions address a central trade-off in online child-safety laws. Age checks can reduce children's access to restricted features, but collecting identity documents or biometric information can create a new privacy risk. New York's statute directs operators to provide a less intrusive route.

 

Background Reading

 

Penalties Reach $25,000 Per Violation

The New York attorney general can seek injunctions, restitution, damages, disgorgement and civil penalties of up to $25,000 for each violation. The law also permits destruction of unlawfully collected data and algorithms trained on that data when a court grants appropriate relief.

 

Individuals who suffer an injury may bring civil actions for damages and other remedies. If a chatbot encouraged harmful conduct and a user later harmed themselves, the law creates rebuttable presumptions that the chatbot caused or contributed to the injury.

 

Contract terms cannot waive or shift liability created by the statute. Courts may also impose joint liability on affiliated companies if a corporate structure was designed to avoid responsibility and would frustrate recovery under the law.

 

The attorney general must operate a public website for complaints and referrals about suspected violations. That mechanism gives enforcement agencies a stream of reports while the rulemaking process develops more detailed compliance standards.

 

Consumer Chatbots Carry the Main Burden

The law includes limited exclusions. A chatbot offered solely for customer service, commercial-product information or account support is outside the main prohibition. Internal systems used by businesses or state and local agencies for employee productivity are also excluded.

 

Those boundaries focus the statute on consumer systems capable of sustained emotional interaction. General assistants may still be affected if they provide prohibited companion features, even when companionship is not their only function.

 

Compliance will require more than filtering a few phrases. Operators must identify New York users, apply age-assurance methods, limit long-term personal memory and test whether a chatbot presents itself as a relationship figure or overrides safety controls to preserve engagement.

 

The 180-day implementation period gives companies time to change product design and verification systems. It also gives the attorney general time to issue rules before enforcement begins, making those regulations the next important test of how broadly New York applies the statute.

 

Image: File photograph of New York Governor Kathy Hochul by Marc A. Hermann/MTA, via Wikimedia Commons. Cropped and resized under the CC BY 2.0 license. The photograph is not from the bill-signing event.