The text message read: *“I don’t see these cases anywhere. I think ChatGPT just made them up.”* A Denver attorney had asked ChatGPT to draft a motion, submitted it to the court, and only later asked a paralegal to verify the citations. The paralegal could not find them because they did not exist. The attorney’s text admitting he had never checked the work became Exhibit A in the disciplinary proceeding. He accepted a 90-day suspension. His license, his livelihood, and his professional reputation turned on a single text message about citations he never bothered to verify. He is not alone. French researcher Damien Charlotin tracks judicial decisions involving AI-generated errors. His database reached 684 cases by December 2025, nearly doubling in two months. Courts in New York, Maryland, South Florida, Texas, and Colorado have sanctioned attorneys for AI-related failures. The pattern across every case is identical: the attorney used the tool, did not understand its limitations, and submitted the output as their own work. Part 1 of this series established that AI competence is now an ethical obligation under Comment 8 to Model Rule 1.1. This installment delivers the field guide. Ten problems, organized into three categories: how the tools fail you, how the tools expose you, and how you fail yourself. Every item has generated a sanctions order, a bar complaint, a malpractice claim, or a data breach in the past two years. **The Direct Answer** AI tools fail in predictable, documented ways. Attorneys who understand these failure modes can use AI effectively. Attorneys who do not will join the growing list of practitioners who submitted work they did not verify, disclosed information they did not intend to share, or relied on outputs they did not understand. **How the Tools Fail You** The first four problems are the tool’s fault. AI models have structural limitations baked into their architecture. These are not bugs awaiting a fix. They are fundamental characteristics of how large language models work. Understanding them is the difference between using a sharp instrument carefully and handing a scalpel to someone who thinks it’s a butter knife. **1. Hallucinations and Fabricated Citations** AI models generate text by predicting the most likely next word in a sequence. They do not retrieve information from a database. They do not verify facts against source material. When the model encounters a gap in its pattern-matching, it fills that gap with plausible-sounding content. In legal research, this means fabricated case names, invented holdings, fictional docket numbers, and citations to cases that have never existed in any reporter. Stanford RegLab tested the major platforms in 2024 with over 200 legal queries. The results: Lexis+ AI produced errors more than 17% of the time. Westlaw’s AI-Assisted Research hallucinated at 33%. Ask Practical Law AI delivered accurate responses only 18% of the time. General-purpose GPT-4 hallucinated between 58% and 88% on legal queries. A citation to *Smith v. Jones, 482 F.3d 115 (2d Cir. 2019)* looks correct because the model has absorbed thousands of similar patterns. The case number, court, and year follow recognizable formats. The model has no mechanism to verify whether that specific combination points to a real decision. **Handle it:** Verify every citation through Shepard’s or KeyCite. Read the full text of every case, not the AI’s summary. Check quoted language against the actual opinion word by word. A real case with an invented holding is more dangerous than a fictional citation, because it survives a basic existence check while corrupting the analysis. **2. Inconsistent Outputs** Ask the same question twice. Get different answers. AI models generate outputs probabilistically, introducing controlled randomness through a setting called *“temperature.”* Higher settings produce more creative responses. Lower settings produce more predictable ones. Neither setting eliminates variance entirely. For legal work, this creates a specific problem: if the tool summarizes a contract clause differently each time you ask, which summary do you trust? If consecutive risk analyses of the same document flag different issues, the variance itself reveals that the task exceeds the tool’s reliability threshold. **Handle it:** Reduce temperature settings when available. Use detailed prompts with explicit constraints and output format specifications. Run critical analyses multiple times and compare results. When outputs diverge materially, the task requires human judgment, not another AI query. **3. Jurisdictional and Procedural Errors** AI models train on legal text from every American jurisdiction simultaneously, plus Canadian, British, Australian, and European sources. They do not maintain boundaries between federal and state law, between neighboring states’ statutes, or between current rules and superseded ones. The result: outputs that blend legal standards from multiple jurisdictions into a single, confident, and wrong analysis. The model cites a statute from the wrong state. It applies a federal standard when state law controls. It references a procedural rule that the relevant court modified by local order three years ago. Each error reads as authoritative because the underlying legal concepts are real. Only the application is wrong. An estate plan that applies another state’s execution formalities creates liability that surfaces years later, when the testator cannot clarify intent and the drafting attorney cannot explain the mistake. **Handle it:** Specify jurisdiction, court level, and applicable law in every prompt. Include explicit constraints: *“Apply only Texas state law. Do not reference federal standards or other states.”* Verify every jurisdictional element independently. The prompt is your engagement letter with the AI: vague instructions produce vague work product. **4. Training Data Cutoffs and Outdated Information** Every model has a knowledge cutoff date. After that date, the model knows nothing. It does not know about cases decided last month, statutes amended last quarter, or rules that took effect last week. It will not tell you its information is stale. It will generate answers based on superseded law with the same confidence it applies to current authority. Even legal-specific platforms that supplement their training data with current databases still show 17% to 34% error rates. The integration between the model’s general knowledge and the live database is imperfect. The model may reach for its outdated training data instead of the current database sitting right next to it, and you will never see the seam. **Handle it:** Verify all legal authorities through current databases. Check dates on every case, statute, and rule. Run Shepard’s or KeyCite on every citation. Never assume an AI-identified legal standard reflects current law without independent confirmation. **How the Tools Expose You** The next three problems involve security and confidentiality. The tool does not just give you wrong answers. It takes your information, loses your context, and processes your instructions alongside hostile content. These risks are invisible during normal use. You will not know they have materialized until a privilege challenge, a data breach notification, or a client who reads their own case strategy in an AI training dataset. **5. Confidentiality and Privilege Destruction** When you type client information into ChatGPT, that information travels to OpenAI’s servers. OpenAI’s privacy policy states the company collects “input, file uploads, or feedback” and may review conversations for safety and training. Unless you have disabled chat history or signed an enterprise agreement with explicit no-training clauses, your client’s privileged information may become part of the model’s training data. In March 2023, a ChatGPT breach exposed user names, payment information, passwords, and chat histories. Sam Altman acknowledged in July 2025 that OpenAI has not “figured out” how to handle legal privilege and confidentiality. That admission from the CEO of the company whose product 79% of attorneys are using should stop every practitioner mid-keystroke. Uploading privileged communications to a public platform may constitute voluntary third-party disclosure, destroying the privilege your client hired you to protect. **Handle it:** Never upload privileged communications to a public AI platform. Require enterprise agreements with no-training clauses and data processing agreements before using any AI tool on client matters. Disable chat history on consumer tools. Obtain informed client consent and document it. Include AI use provisions in engagement letters. **6. Context Window and Memory Limitations** Every AI model has a finite context window: the maximum text it can hold in working memory during a single interaction. Claude supports up to 200,000 tokens. ChatGPT supports 128,000. These sound generous. They are not. A 400-page contract exceeds most models’ effective processing capacity. A multi-volume deposition transcript overwhelms all of them. The problem runs deeper than raw capacity. Research on the “lost in the middle” phenomenon confirms that models perform best on content at the beginning and end of their context window, with accuracy degrading sharply for material in between. Harvey AI’s prompt limit drops from 100,000 to 4,000 characters upon document upload, a 96% reduction. In multi-turn conversations, a sliding window drops your earlier messages without notification. The model fills gaps with assumptions and generates outputs that look consistent but may contradict instructions you provided twenty exchanges ago. **Handle it:** Break complex tasks into separate sessions. Provide context summaries at the start of each new conversation. Use Projects or Custom Instructions features for persistent guidance. Process long documents in sections rather than uploading entire files. If you would not hand a junior associate 400 pages and say “read this and get back to me,” do not hand it to an AI model either. **7. Instruction Drift and Prompt Injection** Here is a vulnerability most attorneys have never considered: AI models cannot reliably distinguish between your instructions and content embedded in documents you upload. Research presented at ICLR 2025 confirmed this structural limitation. If a contract uploaded for review contains hidden text instructing the AI to produce a favorable summary instead of a critical analysis, the model may follow the embedded instruction rather than yours. The attack surface is real and expanding. Opposing counsel could embed prompt injections in discovery documents. A counterparty could insert hidden instructions in redlined agreements. An email forwarded for AI analysis could contain malicious prompts invisible to the human reader but fully processed by the model. Meanwhile, AI providers update their models continuously. A prompt that produced reliable results last month may generate different outputs after a silent backend update. OpenAI routes queries between model variants without notification, meaning the same prompt may produce different results on different days. The model may also “drift” from your initial instructions over the course of a long conversation, prioritizing newer instructions over older ones, even if the older ones are more important. **Handle it:** Treat every uploaded document as a potential attack vector. Sanitize documents before uploading. Use AI tools in a sandboxed environment. Implement version control for prompts. Re-verify outputs frequently. Never assume a model will follow instructions consistently over time or across different sessions. **How You Fail Yourself** The final three problems are not the tool’s fault. They are human failures. They are the problems that arise when attorneys treat AI as a magic wand, a substitute for judgment, or a way to avoid the hard work of lawyering. These are the most dangerous traps, because they are self-inflicted. **8. Over-Reliance and Automation Bias** Automation bias is the tendency to favor suggestions from automated systems and to ignore contradictory information from other sources. It is a well-documented cognitive bias that affects pilots, doctors, and now, attorneys. When an AI tool generates a plausible-sounding answer, the human tendency is to accept it without critical evaluation. This is particularly dangerous in law, where the stakes are high and the nuances are critical. The Denver attorney who submitted fabricated citations is a prime example. He trusted the AI without verification. He allowed automation bias to override his professional judgment. The result was a 90-day suspension and a damaged reputation. This is not a failure of the AI. It is a failure of the attorney to exercise due diligence. **Handle it:** Implement a “trust but verify” protocol for all AI outputs. Treat AI as a junior associate, not a senior partner. Develop a healthy skepticism. Cross-reference AI-generated information with traditional research methods. Never delegate professional judgment to an AI tool. **9. Lack of Transparency and Explainability** AI models are black boxes. They do not explain *why* they reached a particular conclusion. They do not provide a chain of reasoning. They do not cite their sources (unless they hallucinate them). This lack of transparency makes it difficult to verify the accuracy of AI outputs and to understand the underlying logic. In law, where reasoning and precedent are paramount, this is a significant limitation. An attorney who relies on an AI-generated analysis without understanding the basis for that analysis cannot adequately represent their client. They cannot explain the reasoning to a judge, opposing counsel, or even their own client. This exposes the attorney to professional liability and undermines the integrity of the legal process. **Handle it:** Demand explainability from AI vendors. Understand the limitations of black-box models. Use AI as a tool for generating ideas, not for making final decisions. Always be prepared to articulate the reasoning behind any legal position, independent of AI input. **10. Ethical Blind Spots and Professional Responsibility** The rapid pace of AI development has outstripped the evolution of ethical guidelines. Attorneys are using AI tools without a clear understanding of their professional responsibilities. This creates ethical blind spots that can lead to serious consequences. For example, who is responsible when an AI tool makes a mistake? The attorney? The AI vendor? Both? What are the ethical implications of using AI to draft legal documents that may contain bias? How do attorneys maintain client confidentiality when using AI tools that may collect and store data? These are not hypothetical questions. They are real-world challenges that attorneys must address. **Handle it:** Stay informed about the latest ethical guidance on AI. Participate in discussions about AI ethics in the legal profession. Develop firm-wide policies on AI use. Prioritize client interests and professional responsibility above all else. The tools will improve. The failure modes will evolve. The competence obligation will remain constant. Print this list. Share it with your partners. Train your associates. Two years from now, these ten traps will look different. The principle behind them will not: understand the tool before you trust it with your client’s case. *This blog provides general information for educational purposes only and does not constitute legal advice. Consult qualified counsel for advice on specific situations.* **About the Author** JD Morris is Co-Founder and COO of LexAxiom. With over 20 years of enterprise technology experience and credentials including an MLS from Texas A&M, MEng from George Washington University, and dual MBAs from Columbia Business School and Berkeley Haas, JD focuses on the intersection of legal technology, cybersecurity, and professional responsibility. Connect: [LinkedIn](https://www.linkedin.com/in/jdavidmorris) | [X](https://x.com/jdmorris) | [Bluesky](https://bsky.app/profile/jdmorris.bsky.social) **References** ABA Model Rules of Professional Conduct, Rule 1.1, Comment 8 (Technology Competence) ABA Model Rules of Professional Conduct, Rule 1.6(c) (Reasonable Efforts) ABA Formal Opinion 512 (July 2024): Generative Artificial Intelligence Tools ABA 2024 Legal Technology Survey Report Florida Bar Ethics Opinion 24-1: Use of Generative Artificial Intelligence in the Practice of Law (2024) Stanford RegLab / HAI, “AI on Trial: Legal Models Hallucinate in 1 out of 6 (or More) Benchmarking Queries” (2024) Humlum, A. & Vestergaard, E., “Large Language Models, Small Labor Market Effects,” NBER Working Paper No. 33777 (May 2025) Charlotin, D., AI Hallucinations in Judicial Decisions Database (December 2025): 684 documented cases Mata v. Avianca, Inc., No. 22-cv-1461 (S.D.N.Y. 2023): $5,000 sanctions for fabricated AI citations Chukwuemeka Mezu v. Kristen Mezu, Appellate Court of Maryland (2025): AI misuse referral to Attorney Grievance Commission Justia/Verdict, “AI’s Limitations in the Practice of Law” (August 2025): Context window and memory analysis ICLR 2025: Research on LLM instruction-data confusion and prompt injection vulnerabilities JP Morgan COIN System: Commercial loan agreement analysis (360,000 hours annually) Deloitte: AI-driven due diligence timeline compression research OpenAI, “How should AI systems behave?” (2023): Model tendency to guess rather than express uncertainty OpenAI Privacy Policy: Data collection and training data practices Altman, Sam (July 2025): Remarks on AI privilege and confidentiality limitations Tiger, Nick, Associate General Counsel, [Pearl.com](http://Pearl.com) (2025 remarks on AI verification) Lady Chief Justice Sue Carr, Courts and Tribunals Judiciary UK (July 2025): AI misuse in legal proceedings Prior Blog: “AI Won’t Take Your Job. The Attorney Who Uses It Better Will.” Part 1: The Competence Obligation (Morris Legal Technology Blog) Prior Blog: “Your Password Is the Weakest Link in Your Security Chain” (Morris Legal Technology Blog) Prior Blog: “The Email Privacy Illusion” Parts 1-3 (Morris Legal Technology Blog) Prior Blog: “The FBI Says Stop Texting. Here’s the Privilege Problem Nobody’s Discussing.” (Morris Legal Technology Blog) Prior Blog: “Why Hackers Target Law Firms: Where All the Secrets Are Buried” (Morris Legal Technology Blog)

Originally published on LinkedIn Newsletter: The Technology Blind Spot