Lawyer Fined for Submitting AI-Generated Fake Testimony in Court
New Mexico Supreme Court sanctions attorney for unverified ChatGPT brief in murder appeal.
1AI-fabricated evidence in court
A New Mexico defense attorney submitted an appellate brief containing completely fabricated witness testimony and police statements generated by ChatGPT. The filing was for a murder conviction appeal, making the AI's factual errors particularly egregious in a criminal case. This marks one of the most severe documented instances of AI hallucination in legal practice, going beyond previous cases of fake citations.
2Attorney's $5,000 contempt fine
Stephen Aarons was fined and held in contempt by the New Mexico Supreme Court for failing to verify the AI-generated content. The court will also refer him to a disciplinary board. Aarons claimed he used ChatGPT to summarize trial proceedings when taking the appeal in 2022, demonstrating a fundamental misunderstanding of generative AI's limitations in legal contexts where factual accuracy is paramount.
3Judicial skepticism about AI literacy
During an August 21 hearing, justices expressed disbelief at Aarons' lack of awareness about AI's propensity for errors. Justice C. Shannon Bacon pointedly asked if he followed news about widespread reporting on AI hallucination risks. The exchange highlights growing judicial concern about legal professionals' technological competency as AI tools become more accessible.
4Beyond typical citation errors
While dozens of lawyers have faced sanctions for AI-generated incorrect case citations, Aarons' submission contained entirely invented witness accounts. This escalation from reference errors to fabricated evidentiary content raises new ethical questions about AI's role in legal document preparation, particularly in high-stakes criminal proceedings where lives are at stake.
5The 'bulletproof summary' assumption
Aarons told the court he input trial transcripts into ChatGPT expecting a reliable summary, illustrating a dangerous misconception among non-technical professionals about AI's capabilities. His case demonstrates how easily generative AI can be misused by those who don't understand its statistical nature and lack of factual grounding, especially in specialized domains like law.
6Disciplinary board considerations
While expressing remorse, Aarons hopes the disciplinary board will view his actions as an honest mistake rather than intentional misconduct. This case will likely set important precedents for how legal oversight bodies handle similar AI-related ethical violations as the technology becomes more prevalent in legal practice.
7Daily headlines about AI risks
Justice Bacon's remark about AI hallucination stories appearing 'above-the-fold every day' underscores how rapidly this issue has moved from technical concern to mainstream awareness. The legal profession is now forced to confront these challenges head-on, balancing innovation with professional responsibility in an era of easily accessible but unreliable AI tools.
8Broader implications for legal tech
This incident will likely accelerate calls for clearer guidelines about AI use in legal practice. It exposes critical gaps in both technological understanding among practitioners and current ethical frameworks governing emerging tools. The case may prompt bar associations to implement mandatory AI literacy training and stricter verification protocols for computer-assisted legal work.
Original article: Slashdot — Lawyer Fined for Submitting AI-Generated Fake Testimony in Court