Friends, allow me to introduce today's protagonist: Stephen D. Aarons, a New Mexico lawyer who decided to embrace the future of legal practice. He used ChatGPT to draft a murder appeal brief. Efficient? Absolutely. ChatGPT produced 4 entirely fictional witnesses, several fabricated testimonies, and a collection of legal citations that exist in no law library on Earth.

The Court: You Think This Is a Book Launch?

When the New Mexico Supreme Court justices read this brief, they probably went through a predictable emotional journey: confusion ("I don't recognize this case..."), then suspicion ("Let me look that up..."), then fury ("None of this is real.").

The verdict was swift: Aarons was found in contempt of court, fined $5,000, barred from practicing before the New Mexico Supreme Court, and the fantasy-filled brief was stricken from the record entirely. The real victim? His client — accused murderer Oscar Renee Sandoval's appeal was halted, and the court had to appoint a new lawyer to start from scratch.

One man's freedom, delayed by another man's shortcut.

This Isn't the First Time, Friends

If Aarons had done even minimal homework, he'd know that a New York lawyer pulled the exact same stunt in 2023. That attorney also used ChatGPT for legal filings, also got caught with fabricated case citations, also got chewed out by a judge. It was front-page news across the entire legal profession.

But Aarons either didn't notice, or he noticed and thought, "That won't happen to me." This mindset is surprisingly common. Many people use ChatGPT with a quiet sense of immunity: sure, it makes mistakes, but probably not on my task. In low-stakes scenarios, that's harmless. But when you're dealing with a court of law, a murder case, and a person's freedom, false confidence isn't confidence — it's malpractice.

AI Can Be Your Assistant, Not Your Decision-Maker

I'm not here to trash AI in the legal field. In fact, AI has already proven its worth in legal research, document review, and contract analysis. The critical distinction: AI does support work. It doesn't replace judgment.

Having ChatGPT help you organize case logic, compile existing statutes, polish your brief's language — all fine. But asking ChatGPT to "find cases that support my argument" — that's where the disaster starts. When ChatGPT can't find a relevant case, it won't tell you "I don't have one." It'll fabricate one with complete confidence. This isn't a bug — it's the fundamental nature of language models: they generate the most probable next token, not the most truthful next fact.

For all my friends using AI tools, three reminders:

First, every factual claim AI generates — cases, data, names, quotes — must be manually verified. No exceptions. Ever.

Second, if AI can't find what you're looking for but still gives you a polished answer, it's almost certainly making it up. The truly honest response would be "I don't know."

Third, in any scenario with legal, medical, or financial consequences, there must be a human review step. Not "ideally" — mandatorily.

That's all the roasting for today. More tomorrow.