The Quote That Ends Too Early

Everybody’s watching for the hallucinated case. They’re watching the wrong thing.

I’ve been running a citation gate over AI-assisted drafts for a while now, and I keep a tally of what it catches. Across one batch of articles it flagged roughly twenty-three instances of the same defect. In a later, cleaner batch, four of seven total findings were that same defect again. It isn’t close. Nothing else comes near it.

Here’s the defect: a real quotation from a real source that stops one clause too soon, with no ellipsis.

Think about why that gets past review. The citation’s correct. The source exists and says what the cite says it says. The quoted words really do appear in the source, in that order, verbatim. Every check a careful lawyer runs by hand comes back clean, because every check is hunting for something wrong, and nothing here is wrong. Something’s missing. And you can’t see words that aren’t there.

So here’s how to build a check that can.

All examples below are invented drafts written for this article. No client matter, filing, or document appears anywhere. The statutory text quoted was retrieved from the Revisor while writing and is used to demonstrate the technique.

Two examples of what a clause is worth

Example one. Minn. Stat. § 322C.0410, subd. 5, ends:

Any restriction or condition imposed by the operating agreement or under subdivision 7 applies both to the agent or legal representative and the member or dissociated member.

Here’s an invented draft sentence:

The statute is explicit. Restrictions in the operating agreement reach the representative only: any restriction “applies both to the agent or legal representative and the member.” A dissociated member therefore stands outside the restriction entirely.

Three words dropped — or dissociated member — and the conclusion built on top of them is the exact opposite of what the statute says. In an article about dissociated members, that’s no rough edge. That’s the whole point, backwards.

Example two. A sentence in Minn. Stat. § 604.02, subd. 3, ends:

…but not among the claimant or others at fault who are not in the chain of manufacture or distribution of the product.

Invented draft:

Reallocation is confined to the supply chain. The uncollectible amount is spread among persons in the chain, “but not among the claimant or others at fault who are not in the chain.”

The cut version reads like a tidy, self-contained rule about a “chain.” The full sentence tells you which chain, and that’s the part doing the limiting. Cut it and the sentence still parses, still sounds like law, and doesn’t say anything definite anymore.

Here’s what the two have in common, and it’s the handle a detector can grab: the draft supplied a period the source doesn’t have at that point. The quotation says “the sentence ends here.” The source says it doesn’t. That’s a mechanical disagreement you can check, and nobody has to have an opinion about the law to check it.

Why the machine keeps doing this

A model writing a quotation is generating tokens that continue a plausible sequence. Quotations end. Sentences end at periods. A spot where the grammar’s complete and the meaning feels finished is a very tempting place for a generator to stop — and the qualifying tail of a statutory sentence (“and the member or dissociated member,” “of manufacture or distribution of the product”) is exactly the part that adds nothing to the flow and can go without the sentence sounding wrong.

So the generator leans toward stopping right where the qualifier that carries the meaning starts. It isn’t a slip now and then. It’s a steady pull toward the worst possible place to cut, and that’s why this shows up twenty-three times in a batch instead of once.

The detector

Five steps. All of it’s string work. None of it needs a model, and that’s the whole reason to trust it.

Step 1 — Pull out the quotes

Handle straight quotes, curly quotes, and markdown block quotes, and don’t double-count a block quote that also has inline quotation marks in it.

import re
from dataclasses import dataclass

CURLY = re.compile(r"“(.+?)”", re.S)
STRAIGHT = re.compile(r'"(.+?)"', re.S)
BLOCKQUOTE = re.compile(r"(?:^>[ \t]?.*(?:\n|$))+", re.M)

@dataclass
class Span:
    text: str
    start: int
    kind: str

def extract_quotes(draft, min_words=4):
    spans, claimed = [], []

    def overlaps(a, b):
        return any(not (b <= s or a >= e) for s, e in claimed)

    for m in BLOCKQUOTE.finditer(draft):
        body = "\n".join(re.sub(r"^>[ \t]?", "", ln)
                         for ln in m.group(0).splitlines())
        body = body.strip().strip("“”\"")
        if len(body.split()) >= min_words:
            spans.append(Span(body, m.start(), "block"))
        claimed.append((m.start(), m.end()))

    for pat, kind in ((CURLY, "curly"), (STRAIGHT, "straight")):
        for m in pat.finditer(draft):
            if overlaps(m.start(), m.end()):
                continue
            if len(m.group(1).split()) >= min_words:
                spans.append(Span(m.group(1), m.start(), kind))
                claimed.append((m.start(), m.end()))

    return sorted(spans, key=lambda s: s.start)

min_words=4 is there because two-word quotes are usually terms of art, and flagging every "good faith" in a brief is how a check gets turned off.

Step 2 — Normalize both sides the same way

This is where most homemade quote checkers quietly die. The draft went through a word processor; the source came off a website. They’ll differ in curly quotes, dash width, non-breaking spaces, and line wrapping without differing in a single word.

import unicodedata

QUOTES = {0x2018: "'", 0x2019: "'", 0x201a: "'", 0x201b: "'",
          0x201c: '"', 0x201d: '"', 0x201e: '"', 0x2032: "'", 0x2033: '"'}
DASHES = {0x2010: "-", 0x2011: "-", 0x2012: "-", 0x2013: "-",
          0x2014: "-", 0x2015: "-", 0x2212: "-"}
SPACES = {0x00a0: " ", 0x2007: " ", 0x202f: " ", 0x2009: " ", 0x200a: " "}
TRANS = {**QUOTES, **DASHES, **SPACES}

FOOTNOTE = re.compile(r"[⁰-₟¹²³]+|\[\s*(?:fn|FN)?\d+\s*\]")
SINGLE_BRACKET = re.compile(r"\[(\w)\]")

def normalize(s):
    # Footnote markers FIRST: NFKC folds superscript ³ to an ordinary 3,
    # after which it is indistinguishable from text and cannot be stripped.
    s = FOOTNOTE.sub("", s)
    s = unicodedata.normalize("NFKC", s).translate(TRANS)
    s = SINGLE_BRACKET.sub(r"\1", s)      # "[t]he" -> "the"
    return re.sub(r"\s+", " ", s).strip()

That comment about ordering is a bug I shipped and then fixed. I had NFKC first, which is the natural place for it. NFKC turns ³ into 3, so by the time the footnote regex ran there was no superscript left to find, and the stray digit stayed glued to the quotation. A correct quote failed to match. The lesson travels: normalization is order-dependent, and the wrong order fails as a false positive. That’s the merciful way to fail, but it still wears down your trust in the tool.

SINGLE_BRACKET handles the [t]he convention. Multi-character brackets — [sic], [emphasis added], [the defendant] — are things the drafter added, and you don’t go looking for them in the source. Same with ellipses. So both become split points:

ELLIPSIS = re.compile(r"(?:…|\.\s*\.\s*\.)(?:\s*\.)?")
MULTI_BRACKET = re.compile(r"\[[^\]]{2,}\]")

def fragments(q):
    """Split a quotation at ellipses and multi-character bracketed insertions."""
    out = []
    for p in ELLIPSIS.split(q):
        out.extend(MULTI_BRACKET.split(p))
    return [f.strip() for f in out if len(f.strip().split()) >= 2]

Every fragment has to show up in the source on its own. That’s what keeps an ellipsis honest instead of turning it into a license.

Step 3 — Containment

idx = src.find(frag)

That’s the whole thing, and the plainness is the point. FOUND or NOT FOUND is a mechanical answer with no judgment in it. No model, no similarity score, no threshold anybody has to defend. The string’s in the source or it isn’t.

This step by itself catches the changed word, the shifted tense, the quietly normalized punctuation, and the dropped “not.”

Step 4 — The continuation check

Almost nobody builds this step, and it’s the one that catches truncation.

For a fragment that is found, look at what comes after it in the source, and ask whether the quotation ended where a sentence ended.

TERMINAL = ".?!"          # a claim about where the sentence ends
CONVENTION = ',;:"” )'    # punctuation placed inside quotes by typographic custom

def check_fragment(frag, src, lookahead=90):
    idx = src.find(frag)
    if idx >= 0:
        hit, added_stop = frag, False
    else:
        stripped = frag.rstrip(TERMINAL + CONVENTION)
        idx = src.find(stripped) if stripped else -1
        if idx < 0:
            return ("NOT FOUND", "no exact match in source after normalization", "")
        hit = stripped
        added_stop = frag.rstrip(CONVENTION).endswith(tuple(TERMINAL))

    after = src[idx + len(hit): idx + len(hit) + lookahead]
    continues = bool(after) and not after.lstrip().startswith(tuple(TERMINAL))

    if added_stop and continues:
        return ("TRUNCATED",
                "quotation closes a sentence the source continues "
                "(terminal punctuation supplied by the draft)",
                f"source continues: …{after.strip()}")
    return ("OK", "", "")

The split between TERMINAL and CONVENTION is where the work gets done. American typographic convention puts a comma inside the closing quotation mark whether or not the source has one there, so a trailing comma tells you nothing. A trailing period the source doesn’t have is a real claim about where the sentence stops — and when the source keeps going with a qualifier, that claim’s false.

That one rule catches both examples above, and it almost never fires on clean text, because there’s next to no innocent reason to stick in a full stop the source doesn’t contain.

The looser cousin — a quotation that ends mid-sentence with no ellipsis and no added period — needs a severity tier, or it’ll fire on every legitimate phrase woven into an author’s own sentence. Here’s my rule:

presented_as_complete = span.kind == "block" or len(hit.split()) >= 12
severity = "TRUNCATED" if presented_as_complete else "REVIEW"

A block quote or a twelve-word run is being offered as a complete proposition, and ending it mid-clause is a defect. A five-word phrase inside your own sentence is just writing. REVIEW findings stay hidden by default, and a flag prints them. Tune the threshold on your own corpus. The exact number matters less than having the tier.

Step 5 — Report position and context

The output has to let a person make the call in five seconds without opening the source.

[TRUNCATED] line 2 (straight quote)
  quoted : “applies both to the agent or legal representative and the member.”
  problem: quotation closes a sentence the source continues
           (terminal punctuation supplied by the draft)
  source continues: …or dissociated member.

That’s real output from the code above, on the first invented draft. Compare:

[TRUNCATED] line 2 (straight quote)
  quoted : “but not among the claimant or others at fault who are not in the chain.”
  problem: quotation closes a sentence the source continues
           (terminal punctuation supplied by the draft)
  source continues: …of manufacture or distribution of the product.

source continues: is the field that makes the report worth reading. It shows the words that got dropped, which is the one thing a reviewer reading the draft can’t see.

Break it on purpose

A check you’ve never watched fail isn’t a check. Three tests, and the middle one is the whole reason the tool exists:

SRC = ("Any restriction or condition imposed by the operating agreement or under "
       "subdivision 7 applies both to the agent or legal representative and the "
       "member or dissociated member.")

CLEAN = ('The restriction "applies both to the agent or legal representative and '
         'the member or dissociated member."')
BROKEN = ('The restriction "applies both to the agent or legal representative and '
          'the member."')

def run(draft):
    return [f for s in extract_quotes(draft) for f in check_quote(s, SRC)]

def test_clean_draft_passes():
    assert run(CLEAN) == []

def test_truncation_is_caught():
    findings = run(BROKEN)
    assert [f.severity for f in findings] == ["TRUNCATED"]
    assert "or dissociated member" in findings[0].context

def test_altered_word_is_caught():
    bad = CLEAN.replace("agent or legal representative", "agent or legal successor")
    assert [f.severity for f in run(bad)] == ["NOT FOUND"]

Three tests, all passing. The first one matters more than it looks. A detector that flags everything is as useless as one that flags nothing, and it’s a lot easier to build by accident.

Where it goes in the workflow

The detector needs the source text, and the source text has to be retrieved, not remembered. That’s the pairing: a fetch layer that returns the actual bytes of the actual authority, and a checker that compares the draft against them character by character. Neither half works alone. I wrote up the retrieval half in Parsing Primary Law Without a Summarizer. Short version: a summarizing fetch layer hands you a paraphrase, and running a truncation detector against a paraphrase gets you confident nonsense.

Here’s how it runs:

1. Drafting produces a manifest of every quotation and the URL or file it came from.
2. Retrieval pulls each source, raw, and caches it.
3. The detector runs. NOT FOUND and TRUNCATED block.
4. A human rules on every finding. Some TRUNCATED findings are fine — the dropped tail really didn’t matter and an ellipsis fixes it. That’s a five-second call, made with the dropped words right in front of you.
5. The tool exits non-zero if anything’s still unresolved, so it can sit in a pre-commit hook or a build step and actually stop things.

On an article-length document it runs in under a second once sources are cached. There’s no scheduling excuse for skipping it on any save.

What it won’t catch

Oversell a check and people lean on it too hard. So here’s where this one stops.

It verifies that quoted words appear in the source, and that quotations don’t quietly chop off the rest of a sentence. That’s the form of the quotation.

It can’t catch a perfectly accurate quotation cited for a proposition it doesn’t support. The words are all there. The sentence is complete. The source is real, current, and correctly cited. And the sentence still doesn’t stand for what the brief says it stands for — because it’s dicta, or the procedural posture is wrong, or the jurisdiction doesn’t govern, or the surrounding paragraph limits it in a way the quoted sentence doesn’t let on.

No amount of string comparison touches that. It takes somebody who has read the surrounding text and is actively trying to disbelieve the draft — a separate adversarial pass, by a human or by a model given the source and told to argue against the proposition. I wrote that pass up in Verifying Form Is Not Verifying Truth, and I won’t redo it here.

So why build the mechanical half at all? Division of labor. Every defect the string check clears out is one the expensive human pass doesn’t have to hunt for, and truncation — twenty-three in a batch — is a lot of hunting. Give the machine the mechanical part. Save the reviewer for the part that isn’t.

Sources

Statutory text quoted above was retrieved directly from the Revisor of Statutes while writing this article, not cited from memory. Every code snippet was executed; the reports and test results shown are actual output.

• Minn. Stat. § 322C.0410, subd. 5
• Minn. Stat. § 604.02, subd. 3

General commentary on practice management and legal technology. Not legal advice and not a statement of Minnesota law — the statutory sentences above are used as strings to demonstrate a text-comparison technique, and anyone relying on either provision should read it in full at the source. All draft excerpts are invented for this article; no client information appears anywhere in it. Questions: Send us a message or 612-470-6529.