THE TECHNOLOGY BLIND SPOT

Walk into any bar association lunch and the conspiracy theories arrive with the entree. Your “legal AI” is just ChatGPT in a trench coat, charging you $1,200 a month for the trench coat. Hallucinations were solved last year, the vendors will tell you, while the partner across the table will tell you they will never be solved. The vendors are deliberately keeping their products dumb, the theory goes, because the smart version would expose them to unauthorized practice of law claims. The state bars are slow-walking guidance to protect Westlaw and Lexis from new entrants. Pick a theory. Someone at your table believes it.

The theories are wrong. Not because they are paranoid. Because they assume someone is in charge.

That is the part the conspiracy theories get backwards. They imagine vendors with strategies, regulators with frameworks, foundation model labs with internal benchmarks none of us can see. The reality is more uncomfortable. Nobody is at the wheel. The vendors do not know what their products do. The regulators do not know what to require. And the foundation model companies that built the underlying systems disclose, in their own model documentation, that they cannot fully predict the systems’ behaviors.

Theory One: Your Legal AI Is Just GPT in a Trench Coat

The theory: Harvey, CoCounsel, Lexis+ Protégé, and the rest are thin wrappers on OpenAI, Anthropic, and Google models. The “legal AI” branding is marketing. You are paying $1,200 a month per seat for prompt engineering, a logo, and an indemnification letter.

This part is true. Harvey explicitly describes its product as a “multi-model architecture, including models from Anthropic, OpenAI, and Google.” Lexis+ Protégé offers “general-purpose LLMs (GPT-5, Claude Sonnet 4, GPT-4o) within a secure environment.” Thomson Reuters CoCounsel runs on a comparable foundation model stack. The architecture is not secret. The vendors disclose it in their own product documentation. The differentiation is the legal-content layer, the workflow design, and the contractual posture, not the underlying intelligence.

So the conspiracy theorists are correct on the architecture. They are wrong about why it matters. They think the wrapper status is the scandal. It is not.

Underneath the architecture, the same problem applies to wrappers and bespoke models alike. Every prompt Catherine sends through a legal AI tool routes through a chain of subprocessors she has never seen, with confidentiality terms she has never reviewed. [See Your AI Tool Doesn’t Keep Secrets, The Technology Blind Spot (2026).] Whether the underlying model is GPT-5 or a custom legal LLM does not change the routing problem. It does not change the privilege exposure. It does not change the fact that the security pass at her firm’s IT vendor stops at the SOC 2 audit boundary, which ends well before the prompt reaches the foundation model.

Wrapper conspiracy theorists point at marketing. The actual exposure sits in the subprocessor chain, and that chain looks identical for wrappers and non-wrappers. The theorists are aiming at the wrong target.

Theory Two: Hallucinations Were Solved (Or Will Never Be Solved)

The theory has two halves that arrive at opposite conclusions and somehow coexist. Vendors say hallucinations were solved by retrieval-augmented generation. Skeptics say the underlying architecture makes elimination impossible. Both factions believe they have the answer.

Stanford’s RegLab and Human-Centered AI Institute settled the empirical question in 2024, and the peer-reviewed version landed in the Journal of Empirical Legal Studies in 2025. Researchers Varun Magesh and his coauthors tested Lexis+ AI, Westlaw AI-Assisted Research, and Ask Practical Law AI against a preregistered benchmark of more than 200 legal queries. The results: Lexis+ AI hallucinated on 17 percent of queries. Westlaw AI-Assisted Research hallucinated on 33 percent. GPT-4 hallucinated on 43 percent. The legal-specific tools reduced errors compared to general-purpose models. They did not eliminate them.

That finding alone should have settled the conspiracy theory. It did not. The vendors had marketed their products as delivering “hallucination-free linked legal citations” before the Stanford test. After the test, they disputed the methodology rather than publishing their own empirical data. Three years later, neither LexisNexis nor Thomson Reuters has released an independently verifiable measurement of error rates in the products they sell to attorneys at $500 to $1,200 per seat per month.

Nobody is measuring honestly. That is the truth underneath both halves of the conspiracy theory. The “solved” version is vendor marketing absent evidence. The “never solved” version is informed pessimism that no one has tested against current systems. The bars are writing professional responsibility opinions on top of the assumption that somebody, somewhere, has reliable numbers. Nobody has them.

The consequences are not theoretical. Damien Charlotin, a researcher at HEC Paris, maintains a public database of court decisions where attorneys filed AI-generated content containing fabricated citations. The database held 719 cases in January 2026. By late April it had passed 1,300. On March 31, 2026, seventeen U.S. court decisions noted suspected AI hallucinations on a single day. [See The Yes Machine Problem, The Technology Blind Spot (2026).]

Greg Lake walked into the Nebraska Supreme Court in February 2026 to argue a divorce appeal. Of the 63 citations in his brief, 57 contained defects. Twenty were full hallucinations. Four cases he cited, including a Kennedy v. Kennedy he attributed to 2019, did not exist in any jurisdiction. The justices stopped him almost immediately. They asked him how the errors had occurred. Lake explained that he had been on his tenth wedding anniversary, his computer broke while flying, and he uploaded the wrong version of the brief. The court was not persuaded. He later submitted an affidavit admitting he had used generative AI to draft the brief without verifying the citations and called the conduct a “grave error of judgment.” On April 16, 2026, the Nebraska Supreme Court suspended his license. [See Your AI Research Tool Fabricated the Quotation, The Technology Blind Spot (2026).]

The vendors said the problem was solved. The bars wrote opinions assuming the problem was bounded. Greg Lake found out neither was true.

Theory Three: Vendors Are Deliberately Keeping Legal AI Dumb

A third theory cuts in a different direction. Vendors could ship far more capable systems but limit functionality to dodge unauthorized practice of law exposure and product liability. The black box is a moat, the theory goes, deliberately maintained to make accountability impossible.

On its face, this is plausible. UPL exposure is real. Vendors do have legal teams. Limiting capability could be a defensible legal strategy.

It is also wrong. Vendors are not strategically limiting their products. They do not have the internal measurement infrastructure to know what their products are doing in the first place.

Sullivan & Cromwell figured this out the expensive way. On April 9, 2026, the firm filed an emergency motion in the Chapter 15 bankruptcy of Prince Global Holdings before Chief Bankruptcy Judge Martin Glenn in the Southern District of New York. The motion contained fabricated case citations, misquoted authorities, and references to non-existent legal sources. S&C did not catch the errors. Opposing counsel at Boies Schiller Flexner did. Nine days later, on April 18, Andrew Dietderich, founder and co-head of S&C’s global restructuring practice and a Chambers Band 1 partner with nearly three decades at the firm, sent an apology letter to Judge Glenn with a three-page schedule cataloguing every error.

Read what Dietderich actually wrote. The letter describes a firm that had “comprehensive policies and training requirements governing the use of AI tools in legal work.” Two required training modules. Tracked completions. Office Manual language instructing attorneys to “trust nothing and verify everything.” S&C was not the firm that ignored governance. S&C was the firm that built it. The hallucinations got through anyway.

This is the truth that the deliberate-dumbing conspiracy misses. If vendors were strategically limiting their products to manage risk, S&C’s policies would have caught the errors. They did not. The verification chain failed at every level. The vendor did not know enough about its product to flag the errors. The firm did not know enough about the vendor to design a verification protocol that worked. The associate did not know enough about the verification protocol to realize when it had broken. And the partner reviewing the motion did not know enough about the associate’s workflow to identify which sentences had originated where. [See Big AI Is Becoming Facebook, The Technology Blind Spot (2026).]

The black box is not a moat. It is the floor. Nobody has a reliable view of what is happening inside the systems, including the people who built them.

What the Bars Are Actually Doing (And Why the Theories Sound So Plausible)

This is where the theories sound most credible. The state bars have been issuing AI guidance, and several recent opinions look, on first read, sophisticated.

The Oregon State Bar issued Formal Opinion 2025-205 in spring 2025, its general AI guidance. The Washington State Bar issued Advisory Opinion 202505 in November 2025. The Oregon State Bar followed with Formal Opinion 2026-208 in February 2026, addressing autonomous AI agents handling client intake. All three opinions distinguish carefully between machine-based AI, generative AI, agentic AI, and autonomous AI. The taxonomies are clean. The technical vocabulary is current. The opinions cite the same RPC framework attorneys recognize, anchored in competence under Rule 1.1, confidentiality under Rule 1.6, and supervision under Rule 5.3.

Read past the taxonomy and a different picture emerges. The supervisory advice is substantively identical across every category the opinions distinguish. WSBA 202505 tells lawyers to understand their tools, supervise their use, and verify their output. OSB 2026-208, dealing with autonomous AI agents that draft and send client communications without specific lawyer prompting, also tells lawyers to understand their tools, supervise their use, and verify their output. The category distinctions do not produce different supervisory duties. The taxonomy is decorative.

That is the seam. The bars distinguish AI categories at the level of definition because the distinctions are real. They do not distinguish at the level of duty because nobody knows what differential duties for differential technology would look like. The drafters wrote the competence rule for an era when “the technology” meant a fax machine. They wrote the supervisory rule for human assistants. Nobody has updated either rule for systems that produce convincing output their own makers cannot reliably evaluate. The bars are working with the rules they have. The rules cover everything generically because nobody has the data to write rules that cover anything specifically.

There is a limit worth naming. Frontier AI labs do have safety teams, internal evaluations, and red teams. State bars do consult technologists in drafting their opinions. Vendors do have technical leadership who understand more about their products than the marketing materials reveal. The claim is not that no individual anywhere has knowledge of these systems. The claim is that no entity in the chain, neither vendor nor regulator nor firm, holds the comprehensive operationally usable measurement that would convert the conspiracy theory questions into answers. Knowledge exists. It sits fragmented across organizations that do not share data, do not publish numbers, and do not align on definitions. Fragmentation is what produces the empty seat. Expertise alone does not close it.

Catherine should pull her state bar’s most recent AI ethics opinion, or the most recent of her firm’s vendor disclosures, before Thursday. She should read it for one specific question. Where does the technical category collapse into a generic supervisory duty? That paragraph is the seam. It is the place where regulation stops doing actual work, and where the conspiracy theories find their fuel. Once she sees the seam, she stops deferring to the opinion as if it provides operational guidance. It provides taxonomic comfort. The operational gap is hers to close.

The Wheel Keeps Turning

Walk back into the bar association lunch. The theories are still arriving with the entree. Your legal AI is a wrapper. Hallucinations were solved or were never solvable. The vendors are gaming UPL. The bars are protecting the incumbents. Pick the theory that fits your worldview.

None of them is the real story. The real story is that the seat is empty. The vendors are selling products whose error rates they cannot independently verify. The state bars are issuing opinions whose technical categories they cannot operationally distinguish. The foundation model labs publishing system cards are explicit, in their own documentation, that the systems’ behaviors are not fully predictable even to the people who built them. The conspiracy theories assume someone is hiding something. The truth is that the visible part of the system is the system. There is no back room.

That is harder to live with than the conspiracy version. A villain you can confront. A captured regulator you can reform. A wrapper you can replace with a real product. An empty seat is not actionable in any of those familiar ways. It demands something else. It demands that Catherine, and every attorney in her position, accept that she cannot outsource the verification work to the vendor, the regulator, or the firm’s policy document, because none of those entities has the answer she is asking them to provide.

Greg Lake assumed somebody had the wheel. He was on his tenth wedding anniversary. His computer broke. He uploaded the wrong version. None of those things would have mattered if the wheel had been held by anyone else in the chain. It was not. He found out the hard way, in the only courtroom that ever counted, that the seat behind him had been empty the whole time.

About the Author

JD Morris is Co-Founder and COO of LexAxiom, an Agentic AI platform for the business of law. Over a 25-year career, he has built and scaled enterprise technology products across Dell, EMC, VMware, and Cisco, including the first exabyte eDiscovery platform. He holds dual MBAs from Columbia Business School (Finance) and UC Berkeley Haas (Marketing), a Master of Legal Studies in Cybersecurity Law from Texas A&M, and a Master of Engineering from George Washington University. He writes The Technology Blind Spot on the intersection of emerging technology and law. Connect with him on LinkedIn at www.linkedin.com/in/jdavidmorris, on X at @JDMorris_LTech, or on Bluesky at @JDMorris-ltech.bsky.social.

References

Varun Magesh et al., Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools, 22 J. Empirical Legal Stud. 216 (2025).

Damien Charlotin, AI Hallucination Cases Database, https://www.damiencharlotin.com/hallucinations/ (last visited May 7, 2026).

Or. State Bar, Formal Op. 2025-205 (2025), https://www.osbar.org/_docs/ethics/2025-205.pdf.

Or. State Bar, Formal Op. 2026-208 (2026), https://www.osbar.org/_docs/ethics/2026-208.pdf.

Wash. State Bar Ass’n Comm. on Pro. Ethics, Advisory Op. 202505 (2025), https://www.wsba.org/docs/default-source/legal-community/committees/committee-on-professional-ethics/ao-202505.pdf.

Model Rules of Pro. Conduct r. 1.1 (Am. Bar Ass’n 2024).

Model Rules of Pro. Conduct r. 1.6 (Am. Bar Ass’n 2024).

Model Rules of Pro. Conduct r. 5.3 (Am. Bar Ass’n 2024).

Fred Knapp, Nebraska Supreme Court Blasts AI-Authored Court Filings, Recommends Discipline, Neb. Pub. Media (Mar. 20, 2026), https://nebraskapublicmedia.org/en/news/news-articles/nebraska-supreme-court-blasts-ai-authored-court-filings-recommends-discipline/.

Anya Magnuson, Nebraska Supreme Court Suspends Omaha Attorney over AI Use, WOWT (Apr. 15, 2026), https://www.wowt.com/2026/04/16/nebraska-supreme-court-suspends-omaha-attorney-over-ai-use/.

Justin Wise, Sullivan & Cromwell Apologizes to Judge for AI Hallucinations, Bloomberg L. (Apr. 22, 2026), https://news.bloomberglaw.com/business-and-practice/sullivan-cromwell-apologizes-to-judge-for-ai-hallucinations.

Joe Patrice, Sullivan & Cromwell Files Emergency ‘Please Don’t Sanction Us For All These AI Hallucinations’ Letter, Above the Law (Apr. 21, 2026), https://abovethelaw.com/2026/04/sullivan-cromwell-files-emergency-please-dont-sanction-us-for-all-these-ai-hallucinations-letter/.

UC Davis Mabie L. Library, Generative AI Tools and Resources for Law Students, https://libguides.law.ucdavis.edu/genaiforlawstudents/tools (last visited May 7, 2026).

Eugene Volokh, In One Day (Mar. 31), 17 U.S. Court Decisions Noting Suspected AI Hallucinations in Court Filings, Reason: Volokh Conspiracy (Apr. 6, 2026), https://reason.com/volokh/2026/04/06/in-one-day-mar-31-17-u-s-court-decisions-noting-suspected-ai-hallucinations-in-court-filings/.