Five fake case citations. Three fabricated quotations. Five more citations that don't actually support what they claim to. All of them made it from a prosecutor's brief into an official Georgia court order - and nobody caught it until the case reached the state Supreme Court.
The case involves the murder conviction appeal of Hannah Payne. During oral arguments before the Georgia Supreme Court, Chief Justice Nels S.D. Peterson flagged the problem: the trial court's 33-page order denying a new trial was riddled with nonexistent legal citations. Those citations traced directly back to the state's 37-page proposed order, submitted by prosecutor Leslie.
When confronted, Leslie claimed the order had been revised and tried to distance herself from the errors. Chief Justice Peterson wasn't having it. He pointed out that "those nonexistent cases were cited in your initial brief opposing the motion for a new trial" - meaning the hallucinated citations didn't sneak in during some later edit. They were baked in from the start.
The pattern is unmistakable to anyone who has used ChatGPT or similar tools for research: citations that look perfectly formatted but point to cases that simply don't exist, paired with quotations that sound authoritative but were never written by any judge. This is textbook AI hallucination - large language models generating plausible-sounding legal references out of thin air.
This Keeps Happening
This isn't an isolated incident. Since the widely reported Mata v. Avianca case in 2023, where a New York attorney submitted ChatGPT-fabricated citations, courts across the country have been grappling with AI-generated legal fiction. Several jurisdictions now require attorneys to certify that AI tools were not used without human verification, or to disclose AI assistance in filings.
But the Georgia case adds a troubling wrinkle. When a lawyer submits a bad brief, the judge is supposed to be the backstop. Here, the trial court apparently adopted the prosecutor's proposed order wholesale - hallucinated citations and all - without independent verification. That's two layers of human review that failed.
For anyone using AI tools in professional work, the lesson is blunt: AI-generated text that looks authoritative is the most dangerous kind of output. A clearly wrong answer gets caught. A confidently wrong answer that cites sources in perfect format? That sails through review after review until someone actually checks the references. The Georgia Supreme Court happened to check. Most readers won't.