Verifying Form Is Not Verifying Truth: Building a Citation Gate

Everyone in the profession knows the Avianca story by now, and most people have drawn the wrong lesson from it.

On June 22, 2023, Judge P. Kevin Castel of the Southern District of New York sanctioned two lawyers and their firm $5,000 after a brief cited six federal decisions that did not exist. ChatGPT had produced them. Opposing counsel could not find them. Neither could the court. Judge Castel examined the fabricated Varghese opinion and described portions of its reasoning as gibberish. The lawyers were ordered to send corrective letters to every judge whose name had been attached to a fake opinion.

The lesson everybody took: do not let AI write your citations.

The lesson I would draw instead: fabrication is the least dangerous citation failure there is, because it is the only one that announces itself. A case that does not exist gets caught the first time anybody looks it up. It is embarrassing and it is sanctionable, but it is detectable by the crudest possible check.

The failure that should keep you up at night is the one that survives every check you are currently running.

The failure that passes inspection

Picture a citation in a brief. The case is real. It is in the reporter where the brief says it is. The pin cite lands on the right page. The parallel citation is correct. The signal is appropriate. The italics are right. A citation checker — human or software — would pass it without hesitation, because everything checkable about its form is correct.

And the proposition it is cited for is not in the opinion.

Nothing about the shape of that citation is wrong. That is precisely the problem. Every tool most firms use checks shape.

Here is the actual taxonomy, roughly in order of how hard each is to catch:

The top two are what most people mean when they say they “checked the cites.” The bottom six are where briefs actually go wrong, and only one thing catches any of them: somebody goes and reads the source.

Why language models make this specifically worse

It is worth being precise about the mechanism, because “AI makes things up” is too coarse to design against.

A language model produces text that is plausible in context. A citation is one of the most format-constrained, highly patterned artifacts in legal writing — which makes it exactly the kind of thing a model generates fluently. The model has seen a million citations. It knows what one looks like. It will produce one that looks perfect.

It has also seen a million sentences of the form “X v. Y held that [proposition].” Generating a new sentence in that form, with a real case and a plausible proposition, is the same operation. The model is not lying; it has no concept of the difference. It is producing well-formed text, which is what it does.

So the specific danger of AI-assisted legal writing is not sloppiness. It is fluency. The output is more polished than a rushed associate’s and wrong in ways that polish conceals. A citation that is 90% right is far more dangerous than one that is obviously garbage, because the obviously-garbage one gets checked.

And this cuts against the natural workflow. When the draft looks good, you check it less.

What Rule 11 actually requires

This is not merely a quality problem. Minnesota Rule of Civil Procedure 11.02 provides:

“By presenting to the court (whether by signing, filing, submitting, or later advocating) a pleading, written motion, or other document, an attorney or self-represented litigant is certifying that to the best of the person’s knowledge, information, and belief, formed after an inquiry reasonable under the circumstances” …

and, at subsection (b), that

“the claims, defenses, and other legal contentions therein are warranted by existing law or by a nonfrivolous argument for the extension, modification, or reversal of existing law or the establishment of new law.”

Rule 11.03 supplies the consequence: if the court determines Rule 11.02 has been violated, “the court may … impose an appropriate sanction upon the attorneys, law firms, or parties that have violated Rule 11.02 or are responsible for the violation.”

Read the certification carefully. The obligation attaches to the person presenting, and it is measured by the reasonableness of that person’s inquiry. Not the tool’s inquiry. Not the associate’s. Yours. There is no version of this rule under which “the software found the case” is the inquiry.

That is the whole basis for what follows. If you are going to certify an inquiry, you should have a defined one, and it should produce a record.

The four-pass gate

In our practice, nothing with a citation in it goes to a court, to opposing counsel, or to a client as final until it clears four passes. Each pass has a single question and each pass reads the source. None of them may rely on memory, on the drafter’s summary, or on the previous pass.

Pass 1 — Existence. Does this authority exist, at this citation, with these parties, in this court, on this date? Retrieve it. Not “confirm it sounds familiar” — retrieve the text. Anything that cannot be retrieved is not a citation, it is a hypothesis.

Pass 2 — Quotation. Is every quoted string character-for-character identical to the source? This one is mechanical and should be done mechanically: string comparison catches the changed word, the silently normalized punctuation, the dropped “not.” Pay specific attention to ellipses and brackets. An ellipsis that removes a qualifier can reverse a holding while remaining, technically, an accurate quotation.

Pass 3 — Proposition. This is the pass that matters and the one nobody runs. Does the source actually stand for what the brief says it stands for? Answering it means reading the surrounding discussion, not the headnote — headnotes are not the opinion and are frequently the origin of a misattribution. Distinguish holding from dicta. Check that the procedural posture supports the use: a case reversing a grant of summary judgment does not establish what the facts were. Check the jurisdiction actually governs the proposition.

Pass 4 — Currency. Is it still good law? Reversed, vacated, superseded by statute, limited by later authority, or — the quiet one — a rule that has been renumbered or abrogated since you last used it. This pass is time-dependent in a way the others are not: a citation verified two years ago is not verified today.

The output is a manifest: every authority in the document, what was checked, what was retrieved, and the disposition. Verified, corrected, or removed.

The design rules that make it work

Having the four passes written down is not the same as having a gate. Four things make it real.

1. The verifier is not the drafter. Whoever built the argument is the worst possible person to check whether the authority supports it, because they already believe it does. In our chain this is a structural separation: the drafting specialist hands off to a citation specialist whose entire job is to disbelieve the draft. If you are a solo, the equivalent is a separate session with a fresh context that has not seen your argument — hand it the document and the sources, not your reasoning.

2. It is a step, not a habit. Habits fail on the day you are in a hurry, which is the day the brief is due. Make verification a required stage between drafting and formatting, so that skipping it means noticing you skipped it.

3. No citing from memory, ever — by anyone. The drafting instruction in our practice is explicit: the drafter may not produce a citation from recollection. Either the authority was retrieved and read in this session, or it goes in as a placeholder marked for research. This eliminates the entire fabrication category before verification even starts, because there is nothing to fabricate from.

4. Unverifiable means removed, not flagged. A citation that cannot be confirmed comes out of the document. Not highlighted in yellow for later. Out. Flags get lost between the draft and the filing; an argument that has lost its support should be visibly missing its support, so somebody has to deal with it.

What it costs

It is slower. There is no version of this that is not slower, and anybody selling you a citation checker that is both instant and does Pass 3 is selling you Pass 1 with a nicer interface.

But the accounting is not close. A brief with thirty citations takes perhaps ninety minutes to run properly through all four passes — less if the drafting rule above is enforced, because most of Pass 1 was already done during drafting. Against that: the cost of citing a case for something it does not say is a court that stops trusting your representations, an opponent who now leads their reply with your error, a client whose motion turns on an argument that has quietly lost its foundation, and — in the bad case — a Rule 11 proceeding.

The asymmetry is the argument. This is cheap insurance against an uncapped loss, and it is the single highest-leverage process control I know of in a practice that uses AI at all.

There is a second benefit I did not anticipate. Pass 3 improves the brief. Forcing somebody to articulate, for each authority, exactly what it stands for and why that supports this proposition, surfaces arguments that are weaker than they looked. Several times the verification pass has sent a section back for rewriting — not because a citation was wrong, but because the honest answer to “what does this case actually give us” turned out to be “less than we implied.” Better to find that out from your own reviewer than from the bench.

Sources

• Mata v. Avianca, Inc., 678 F. Supp. 3d 443 (S.D.N.Y. 2023) (sanctions order entered June 22, 2023)
• Minn. R. Civ. P. 11.02 and 11.03

Rules quoted above were verified against the Revisor of Statutes rather than cited from memory. General commentary on practice management; not legal advice, not ethics advice. No client matter is described in this article.