What Happened
In a personal injury lawsuit (Mata v. Avianca) filed in the Southern District of New York, attorney Steven Schwartz of the firm Levidow, Levidow & Oberman used ChatGPT to research relevant case law. The AI generated six case citations that appeared legitimate, complete with realistic case names, docket numbers, and judicial reasoning, but none of them existed. Cases like “Varghese v. China Southern Airlines” and “Martinez v. Delta Air Lines” were entirely fabricated.
Schwartz submitted a 10-page brief containing these hallucinated citations. When opposing counsel and the court could not locate the cases, Judge P. Kevin Castel ordered Schwartz to produce copies. Schwartz then asked ChatGPT to verify whether the cases were real, and ChatGPT confirmed they were. It was only after extensive failed searches that Schwartz realized the cases were fabricated.
Why It Matters
This incident became the most widely reported example of AI hallucination causing real-world professional harm. It demonstrated that ChatGPT generates confident, detailed, but entirely fabricated information indistinguishable from real citations without independent verification. The case exposed a critical vulnerability: professionals in high-stakes fields trusting AI outputs without verification. It accelerated the adoption of AI usage policies across the legal profession and prompted multiple bar associations to issue guidance on AI in legal practice.
Lessons Learned
AI-generated content must be independently verified before use in any professional or legal context. ChatGPT’s hallucinations are not random gibberish but structured, plausible fabrications that mimic real outputs. Even asking the AI to verify its own outputs is unreliable, as it will confidently confirm fabricated information. Professional fields require clear protocols for AI tool usage, including mandatory human verification steps. The duty of competence in legal practice extends to understanding the limitations of tools used in research.
Current Status
Judge Castel fined Schwartz and LoDuca $5,000 each in June 2023. The incident prompted law firms worldwide to implement AI usage policies. Multiple courts have since required attorneys to disclose AI usage in filings. Legal AI tools like Westlaw and LexisNexis developed features specifically designed to prevent citation hallucination. The case is taught in legal ethics courses as a cautionary example.