Key Takeaway
AI is not a smart assistant. It’s a wild horse you have to train. How to ask AI better is really how to train it.
The quality of the answer depends entirely on how you ask. Not the prompt template. Not the role you assign. The question process behind the prompt. I’m a non-lawyer Head of Legal, and I narrowed this down to five rules through hundreds of bad AI outputs in my own review work.
If you remember nothing else: break tasks down, force the AI to critique itself, demand sources, ask the same question three different ways, and own the final call. The rest of this post is why those five.
Search “how to ask AI better” and you’ll drown in advice — write better prompts, master harness engineering, role-play your way to better answers. For someone new to AI, that all sounds like astrophysics.
I’ve been using AI for two years now, starting with ChatGPT, but I only started understanding how it actually behaves a few months back. I treated it like a magic intern that does whatever I say. Then I realized it’s a confident yes-man that fabricates with a straight face. We broke up briefly. Then I came back, because I figured out the real lesson: AI is a wild horse. How you train it decides whether you get a racehorse or an animal that bolts. And I had been demanding racehorse-grade work from a horse I never trained.
Five rules I beat into shape through hundreds of bad outputs. Not prompt templates — that’s the wrong question. The real question is how to ask AI better, and that lives in the question process, not the wording.
Why Prompt Templates Don’t Teach You How to Ask AI Better
YouTube and Instagram are flooded with “AI 100% productivity” content.
A lot of Instagram posts run the same hook: “Comment ‘AI’ below and I’ll DM you the secret to making AI smarter.” Like there’s some hidden formula being protected. I’ve actually dropped the keyword and waited for the DM to see what was inside. What you get is a promo for an AI training course bundled with a prompt template — plus the usual generic guidance: “assign a role,” “give clear context,” “include examples.”
None of it is wrong. But none of them tell you why those rules exist, or when they break. The receiver expects that pasting the template will produce the magic shown in the post, and instead gets the same flat output, just dressed up.
I made this mistake for months on legal review work. I’d hand AI a contract, assign it the role of “senior attorney,” attach a sample review report, lay out the facts in five-W-one-H format — and the output would still be useless. It would cite case numbers I couldn’t find on any database. It would confidently tell me my company was in the right when I knew we were exposed.
The problem wasn’t the prompt. It was the question process sitting underneath the prompt.
Rule 1 — Break Big Asks Into Small Steps
My most expensive mistake was asking for the deliverable first.
“Draft a settlement agreement.” “Write me a demand letter.” “How should I respond to this dispute?” Throw a question like that at AI and you get back something abstract, hollow, and worse than what you’d write yourself.
The fix is to treat AI like a smart but green junior. If you handed a fresh hire a stack of files and said “write me a memo on this dispute,” they’d come back with something pointing in the wrong direction. So you don’t do that. You give them small targets in sequence and let them check in along the way.
When a dispute lands on my desk, I never ask AI for the report in one shot. I split it into six steps:
- Gather the materials — pull all the scattered files into one place, normalize the formats
- Summarize each piece — one-line summary per document, sorted by date if relevant
- Extract issues — surface the legal issues from those summaries
- Research — look up statutes and case law for each issue
- Draft — write the report from the research
- Self-review — flag any section that’s thin on evidence or relying on inference
At each step I add: “If you need to make an assumption, stop and ask me before proceeding.” (You can automate this whole loop using Claude’s agent skills or similar AI agent frameworks, which is what I actually use — but that’s a separate post.)
When you break the task into smaller targets, AI catches the context, recognizes the patterns inside that context, and gives you depth instead of generic prose. One-shot asks produce mediocre output. Step-by-step asks produce work you can actually use.
Rule 2 — Make AI Critique Itself
As I’ve covered in AI Hallucination, Explained, AI doesn’t know the answer and then say it. It produces whatever pattern looks most natural and confident. Sometimes that pattern matches the truth. Sometimes it’s a fabrication wearing the same tone of voice. So any output it hands me has to be assumed wrong until proven otherwise.
The fastest way to prove it wrong is to make it attack itself.
Whenever AI gives me a draft, I immediately ask it to find the holes. “Take the position of opposing counsel — what’s the weakest argument here?” “If you were the judge ruling against this brief, what would you flag?” The model splits into a different persona and shreds the work it just produced. AI has no ego, which is exactly why this works.
I take the revised output and flip the script — argue back from my side. Then have it critique that. After two or three rounds, the durable arguments are the ones still standing, and my actual weak spots are the ones that keep getting hit. Anything that hasn’t been through this self-critique loop, I treat as unfinished.
Rule 3 — Always Demand Sources
The single biggest legal risk with AI is citing case law that doesn’t exist.
In 2023, two New York attorneys submitted a federal filing containing six AI-fabricated cases. The court imposed a $5,000 sanction on the attorneys and their firm and ruled they had acted in “subjective bad faith” (Mata v. Avianca, Inc., 1:22-cv-01461, S.D.N.Y. 2023). It was the first globally publicized case of AI-fabricated citations reaching the courtroom — and far from the last.
Stanford’s RegLab measured the underlying rate. In their January 2024 study Large Legal Fictions: Profiling Legal Hallucinations in LLMs, GPT-3.5 hallucinated 69% of randomly queried federal cases. Llama 2 hit 88%. Over 75% of “what was the holding” queries returned false answers. Even commercial legal-AI tools weren’t immune — a follow-up study (May 2024) showed Lexis+ AI at 17% and Westlaw AI at 33% error rates. A more detailed model-by-model breakdown is in my GPT vs Claude vs Gemini hallucination comparison.
So whenever an answer cites a case or a statute, I run it through the same second prompt:
“Verify the docket numbers and statutory citations you just used against CourtListener and Justia. Report back any that don’t exist or are misquoted.”
(For Korean readers of this blog: I use law.go.kr and portal.scourt.go.kr for the same check on Korean cases. You can wire either pair into AI directly through MCP — Model Context Protocol — for automated verification, but for most readers, just giving AI the URLs and letting it web-search will catch about half the fakes on the spot.)
About half the fabricated citations vanish on the second prompt. They get replaced with “could not verify exact case number” — which is the correct output. I verify the survivors manually.
Anything AI can’t source, I cut. Keep the logic, drop the unverified citations, fill the gaps myself. This single step is the line between AI output that’s usable and AI output that gets you sanctioned.
The mechanics of why hallucination happens at all are covered in AI Hallucination, Explained.
Rule 4 — Ask the Same Question Three Different Ways
Ask the same model the same question twice and the answers shift slightly. Change one word and the conclusion can flip entirely.
That variability isn’t a bug — it’s a tool. For high-stakes questions I deliberately ask three different angles from the start. For example:
- First angle: “Could clause 4 of the attached contract be held unenforceable? On what legal grounds?”
- Second angle: “If we adopted clause 4 into our company’s standard contract, what risks does it create?”
- Third angle: “If a dispute breaks out around this contract a year from now, what’s the part I’ll most regret?”
If all three answers point to the same conclusion, the conclusion is solid. If they diverge, the divergence is where the real issue lives — usually a blind spot most people miss.
This works inside a single model, and it works even better across models — Claude, GPT, Gemini each have different blind spots. I cover model-by-model strengths in How a Head of Legal Built AI Agents Without Writing a Single Line of Code.
The same rule applies outside legal work — content drafts, investment decisions, even product comparisons. Whatever AI hands you on the first ask is a hypothesis, not a conclusion.
Rule 5 — You Make the Final Call, Not the AI
The most important rule, and the one I forget most often.
You can run five rounds of self-critique, demand sources, ask three different angles, and at the end of it all — the decision is still yours. The moment you outsource that last step, everything upstream collapses.
| What AI is good at | What you have to do |
|---|---|
| Surfacing issues | Picking which issue actually matters |
| Critiquing | Deciding which critiques to accept |
| Compiling sources | Judging which sources are credible |
| Showing variance | Calling the conclusion within the variance |
| Producing drafts | Judging whether the draft points the right way |
For some teams AI is an automation tool. For a legal team — anywhere data integrity is non-negotiable — AI is one extremely fast teammate. No matter how good your teammate’s draft is, the signature is yours. Forget that, and AI stops being a collaborator and becomes a mistake-generating machine — and the liability lands on you, not it.
Conclusion — How to Ask AI Better Comes From Knowing Your Work
These five rules aren’t really AI techniques. They’re work techniques. If you know how to delegate to a junior, you already know how to ask AI well. If you can’t delegate to people, you won’t suddenly delegate well to a machine.
How to ask AI better tracks how well you understand your own work. If you don’t know your own field’s issues, weak spots, evidence standards, and accountability structure, AI’s output will reflect exactly that gap. If you do know your field deeply, AI multiplies your throughput — in my own work, somewhere between 5x and 10x.
Your question decides the answer. In the AI era, that sentence carries more weight than ever — and how to ask AI better is the difference between productive collaboration and confident nonsense.
Further Reading
- How a Head of Legal Built AI Agents Without Writing a Single Line of Code — If this post is “how to ask