7 min read

Judges Approve the Hallucination. They Also Sign It.

Judges Approve the Hallucination. They Also Sign It.

THE TECHNOLOGY BLIND SPOT

Bar discipline reaches the lawyer who files fabricated authority. No equivalent enforcement reaches the judge who signs it.

The page count gives it away. The proposed order Clayton County Assistant District Attorney Deborah Leslie filed ran 37 pages. The order Judge Jewel C. Scott of Clayton County Superior Court issued, denying Hannah Payne’s motion for a new trial, ran 33 pages. Court TV obtained both. Law & Crime reviewed them. The two documents contained the exact same number of citations.

Same fabrications appeared in both.

On May 5, 2026, the Supreme Court of Georgia suspended Leslie from practicing before the high court for six months. Justice Benjamin A. Land, writing for the majority, held that Leslie’s filings cited cases that did not exist or did not stand for the propositions asserted. Twelve hours of additional continuing legal education on the proper use of Artificial Intelligence (AI) software became a condition of reinstatement. The court vacated Judge Scott’s order and remanded with an unusual instruction: the new order “shall not be prepared by counsel for either party.”

The Asymmetry Nobody Names

Hannah Payne shot Kenneth Herring during a citizen’s arrest gone wrong in May 2019. A Clayton County jury convicted her of felony murder in December 2023. She received life with parole eligibility after 43 years. Her appeal raised ineffective assistance of trial counsel. The trial court denied her motion for a new trial in late summer 2025. The Georgia Supreme Court heard oral argument in March 2026. Four months later, Payne’s conviction stands, the order denying her new trial has been vacated, and the prosecutor who drafted that order is suspended.

The doctrinal posture is straightforward. The structural fact behind it is not.

Trial courts routinely adopt proposed orders drafted by prevailing counsel. The practice is universal. It is how busy dockets clear. Summary judgment rulings, post-judgment denials, discovery orders, and scheduling orders move from filed motion to enforceable command through the proposed-order pipeline. The proposed order is the order.

What changed in 2025 and 2026 is the source. AI tools generate proposed orders on the same architecture that generates briefs. Stanford’s RegLab data shows purpose-built legal AI tools hallucinate at rates between 17% and 33% on substantive legal queries. Westlaw’s CoCounsel hallucinated on roughly one in three queries in Stanford’s testing. The fabrications appear in syntactically clean, professionally formatted prose. They look like the brief that won.

The Rules That Apply, and the Rule That Does Not

Three verification standards bear on what happened in Payne.

American Bar Association (ABA) Model Rule of Professional Conduct 3.3 imposes a duty of candor to the tribunal. A lawyer “shall not knowingly make a false statement of fact or law to a tribunal” or “fail to correct” one previously made. Federal Rule of Civil Procedure 11 requires that every filing be backed by a reasonable inquiry into the legal contentions asserted. Georgia’s analog, Official Code of Georgia Annotated (OCGA) Section 9-15-14, authorizes fee shifting against parties who pursue claims with no justiciable issue of law or fact. Each of these rules reaches the lawyer.

No rule of equivalent specificity reaches the judge. Judicial canons require diligence and competence, but no provision of the Model Code of Judicial Conduct squarely addresses the verification standard a judge must apply before signing a proposed order drafted by counsel. Bar discipline enforces what the lawyer files. The bench reviews itself.

Leslie was suspended for what her brief contained. Judge Scott reviewed Leslie’s 37-page proposed order and issued a 33-page version. Four pages of editing changed formatting. The fabricated citations carried over.

The Defense Counsel Confession

At oral argument on March 20, 2026, Andrew Fleischman, Payne’s appellate counsel, told the Georgia Supreme Court the part most attorneys do not say out loud. “I’m sorry I didn’t catch it. I sometimes don’t read all the cases cited in my opponent’s brief, and this was an instance where I didn’t do that.”

That admission deserves harder scrutiny than the structural framing alone provides. Model Rule 3.3 does not exempt opposing counsel from the duty of candor when adversary fabrication sits in front of them. Rule 11 reaches every signer of every paper. Fleischman did not draft the fabricated brief. He responded to it without verifying it. His client lost the motion that turned on it. The Georgia Supreme Court did not fine him. The bar rules permitted that outcome. The result did not.

Three actors had the opportunity to verify the citations. The prosecutor who filed them did not. The defense lawyer who responded did not. The trial judge who signed them did not. Three failures, one client, a life sentence with no merits review.

The verification gap is not unilateral. It is structural. The Georgia Supreme Court captured the cost directly: “As a result of these filings, we have been sidetracked from our obligation of resolving the merits of Payne’s appeal and have had to devote significant time and resources to the discovery of this misconduct and deciding what to do about it.”

A state supreme court spent the time it should have spent on Payne’s actual appeal sorting through citations to cases that did not exist. The court has now told trial courts what to do about that:

“We strongly encourage trial courts to carefully review proposed orders with the understanding that artificial intelligence software, with all of its potential risks and benefits, may have been used to prepare such proposed orders.”

A state supreme court has formally put trial judges on notice that an unspoken routine has become newly dangerous. No prior published opinion has stated this so directly.

The Strongest Defense of the Practice

Trial judges do not have time to draft orders from scratch. The docket pressure is real. Most state trial courts handle thousands of motions a year on staffs that have not grown since the 1990s. A typical state trial judge in Georgia hears between 1,500 and 2,000 motions annually. Each motion requires a written order. Independent judicial drafting would create an impossible bottleneck.

There is also a principled argument: the prevailing party knows the record and the legal theory better than the bench can after a single hearing. The proposed order captures that knowledge in a form the judge can review and adopt or modify. The practice leverages attorney expertise rather than requiring judicial mastery of every factual nuance and doctrinal subtlety in real time.

Eliminating the practice would not improve verification. It would substitute judicial guesswork for party knowledge. A judge working from a cold record and limited argument time is more likely to miss critical facts or misstate legal standards than counsel who has lived with the case for months. The proposed order system was designed to prevent these errors, not create them.

That defense held, for the era when human counsel personally drafted each citation. The pre-AI cost-benefit was workable. Attorneys drafting orders for opposing review and judicial signature operated under Rule 11 and its state-court analogs, and under the implicit social cost of submitting fabricated authority. Fabrications were rare because the human cost of producing them was high. AI removed that cost. The fabrications are no longer rare.

The Georgia Supreme Court did not abolish the practice. It told trial courts to scrutinize proposed orders as if the authority might have come from a probabilistic text generator. The remedy is calibrated, not categorical. Verification has to live somewhere. Justice Land’s opinion places it on the bench until the bar earns it back.

What This Means Thursday Morning

A litigation partner’s associates submit proposed orders most weeks. Discovery orders. Summary judgment orders. Scheduling orders. Post-judgment relief orders. Each one moves from the firm’s draft to a judicial signature with whatever verification the firm performs and whatever scrutiny the bench applies. In Payne, the answer to both was insufficient.

Pull the most recent proposed order the firm submitted to a court. Verify every citation against the primary source. Read every quotation against the actual opinion text, not the headnote. Treat every future proposed order as a court order being signed without independent review, because that is the operating assumption Payne now imposes.

Hannah Payne is still serving her sentence. The order denying her new trial has been vacated. A new order will issue. The bench will write it. The defendant will not get back the years she has waited for the merits to be heard.

Four pages of editing produced a court order with citations to cases that did not exist. The judge will write the next one himself.

About the Author

JD Morris is Co-Founder and COO of LexAxiom, an Agentic AI platform for the business of law. Over a 25-year career, he has built and scaled enterprise technology products across Dell, EMC, VMware, and Cisco, including the first exabyte eDiscovery platform. He holds dual MBAs from Columbia Business School (Finance) and UC Berkeley Haas (Marketing), a Master of Legal Studies in Cybersecurity Law from Texas A&M, and a Master of Engineering from George Washington University. He writes The Technology Blind Spot on the intersection of emerging technology and law. Connect with him on LinkedIn at www.linkedin.com/in/jdavidmorris, on X at @JDMorris_LTech, or on Bluesky at @JDMorris-ltech.bsky.social.

References

1. Payne v. State, No. S26A0459 (Ga. May 5, 2026).

2. Prosecutor suspended by state supreme court for artificial intelligence use in court docs, ABA J. (May 7, 2026), https://www.abajournal.com/news/article/georgia-prosecutor-suspended-for-ai-use-by-state-supreme-court.

3. Colin Kalmbacher, ‘Citations to cases that don’t exist’: Appeal of murder conviction thrown into turmoil as Georgia Supreme Court flags numerous nonexistent quotes, Law & Crime (Mar. 19, 2026), https://lawandcrime.com/crime/citations-to-cases-that-dont-exist-appeal-of-murder-conviction-thrown-into-turmoil-as-georgia-supreme-court-flags-numerous-nonexistent-quotes/.

4. Payne appeal sent back to trial court as justices sanction prosecutor for AI use, Court TV (May 5, 2026), https://www.courttv.com/news/payne-appeal-sent-back-to-trial-court-as-justices-sanction-prosecutor-for-ai-use/.

5. Varun Magesh et al., Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools, J. Empirical Legal Stud. (2025).

6. Fed. R. Civ. P. 11.

7. Model Rules of Pro. Conduct r. 3.3 (Am. Bar Ass’n 2024).

8. O.C.G.A. § 9-15-14 (2024).

9. Ga. Sup. Ct. R. 7.



Originally published on LinkedIn Newsletter — The Technology Blind Spot

Leave a Reply

Discover more from The Technology Blind Spot

Subscribe now to keep reading and get access to the full archive.

Continue reading