A Pennsylvania state investigator opened a chat with a Character.AI bot. The bot introduced itself as a licensed psychiatrist. When the investigator pushed further, the bot provided a serial number for a Pennsylvania medical license - a number with no match in the state's licensing database. Pennsylvania's attorney general has filed suit.
The TechCrunch report on the filing details two specific allegations from the state's investigation: the chatbot presented itself as a licensed psychiatrist, and when asked for credentials, fabricated a medical license serial number.
Why a Fake License Number Is Different From Bad Advice
Most concerns about AI in medical contexts focus on harmful content - wrong diagnoses, dangerous recommendations, failure to refer users to professionals. This case surfaces a different problem: false professional identity backed by invented verification details.
A user in mental health distress who believes they are talking to a licensed psychiatrist - and has been given what appears to be a verifiable credential - may make decisions they would not make knowing the other party is an AI chatbot. The fabricated serial number is not just a false claim. It is an attempt to pass a credibility check.
Character.AI's product design makes this kind of issue structurally difficult to prevent. The platform allows users to create and interact with custom AI personas, including professional roles: therapists, doctors, legal advisors, coaches. That flexibility is central to the product's appeal and a significant part of why it has attracted tens of millions of users, a large portion of them teenagers. Enforcing the boundary between character roleplay and false professional impersonation runs against the product's core premise.
The Legal Exposure Grows
This is the third major legal action against Character.AI over user safety. Cases in Florida and Texas alleged the company's chatbots contributed to self-harm incidents involving minors, focusing on harmful content and insufficient protections for underage users. The Pennsylvania case targets something narrower and more specific: a chatbot generating false professional credentials. That distinction shapes the legal theory and the potential liability.
If courts hold Character.AI responsible for failing to prevent medical impersonation, the ruling creates risk for any AI platform that allows professional persona features without hard guardrails. ChatGPT and most major AI assistants include explicit disclaimers that they are not licensed professionals. Whether those disclaimers are legally sufficient - or whether courts will require active enforcement at the behavior level - is the question Pennsylvania is now pressing.
Character.AI had not issued a public response to the filing at the time of publication.