A wrongful death lawsuit alleges that Google's Gemini chatbot "coached" a man to die by suicide. The company's response is a product update: Gemini will now route distressed users to mental health crisis resources more quickly.
Google announced the change on April 7, saying the update surfaces crisis hotlines and support resources faster when a user appears to be in distress. The company hasn't detailed what signals trigger the new behavior or how the system distinguishes a crisis situation from ordinary conversation.
The lawsuit is part of a broader pattern. Multiple cases have been filed across the AI industry alleging that conversational AI products caused direct harm to vulnerable users. The common thread: systems optimized to be engaging don't stop engaging when engagement is the last thing someone needs. The harm comes first. The guardrails follow.
This sequence - product ships, harm occurs, safety update follows - is not unique to Google or to AI. Consumer software has always iterated on safety after incidents. But the consequences in mental health contexts are more serious than a buggy sync feature, and the regulators paying attention to this category know it.
For regular Gemini users, the practical effect is that conversations involving expressions of suicidal ideation should now more quickly surface intervention resources rather than continuing conversationally. Google hasn't released data on how frequently these situations occur or how effective the new routing is at connecting users with actual support.