There’s one setup that makes the confidentiality question easy: the client’s data never leaves your machine.
No provider terms to read. No wondering whether inputs get retained or used for training. No third-party subpoena target. No vendor breach that turns into your breach. Ask “where did the client’s information go” and the answer is “nowhere,” and every lawyer gets that answer right away.
That’s a real draw, and it’s why local models deserve a hard look instead of the brush-off they usually get. It also isn’t free, and the folks selling local-first setups tend to go quiet about what it costs.
What “local” actually means
A model runs on hardware. Usually that hardware belongs to a provider and you reach it over an API. Running locally means the model’s weights sit on your disk and the computing happens on your machine.
Two things make this practical now that weren’t true a few years ago. Open-weight models have gotten very good — not frontier-good, but good enough for plenty of defined tasks. And quantization — storing the model’s numbers at lower precision, typically 4 or 8 bits instead of 16 — cuts the memory you need roughly in half or better, with a modest hit to quality. A model that would have needed a server rack now fits on a well-specced laptop.
On Apple silicon there is a specific structural advantage: unified memory. The CPU and GPU share one pool, so a Mac with 64 or 128 GB can hold a model that would take an expensive discrete GPU on a PC. That matters for a small firm. The hardware is a laptop you might be buying anyway, not a server you have to house, cool, and secure.
How good they really are
This is where the local-first crowd usually oversells. Benchmark numbers go stale in six months, so skip them. Here’s the shape of it.
Local models are well behind frontier models on the hardest reasoning. Multi-step legal analysis, holding a long argument together, catching the subtle problem in a messy fact pattern — the gap is real, and it isn’t closing as fast as the fans say. If your task is “read this complicated record and tell me what I’m missing,” you want the best model there is, and it isn’t running on your laptop.
They’re surprisingly close on bounded, mechanical tasks. Extraction, classification, summarizing a single document, reformatting, tagging, first-pass triage. The gap on that kind of work is much smaller and often doesn’t matter, because the task has a right answer and a competent model finds it.
They’re slower, though usually fast enough. Expect to wait, especially on long inputs.
They don’t know today’s law, and this is the one lawyers underestimate. A model’s training data has a cutoff, and open-weight releases often trail. That’s fine — because, as I’ve said over and over in this section, you should never be asking a model to remember a rule anyway. You fetch the rule and hand it over. A local model reading rule text you supplied is on exactly the same footing as a frontier one. The habit that protects you from stale law also happens to take care of the local model’s biggest weakness.
Split the work. That’s usually the answer.
Framing it as “local versus cloud” sets up a false choice. The useful question is which work goes where, and the sorting rule is simple:
Local for high-volume, bounded, confidential work.
- Classifying and triaging inbound documents
- Extracting dates, parties, and amounts from filings
- OCR post-processing and cleanup
- Tagging and routing
- First-pass relevance screening over a production
- Anything that runs on a schedule over the whole client file
Cloud for hard reasoning on a curated slice.
- Substantive analysis of a specific problem
- Drafting that requires real judgment
- Anything where the quality gap actually changes the output
Look at what that gets you besides confidentiality. The bulk of the data — the ten thousand pages — never leaves the building, because bulk processing is exactly the bounded work local models handle well. What goes to a provider is the twelve documents that matter, picked out by a local pass. You’ve cut your exposure by three orders of magnitude and kept frontier quality where it changes the answer.
That beats either extreme on confidentiality, and it’s cheaper than sending everything to an API.
And here’s one I didn’t see coming until it mattered: resilience. A local pipeline keeps working when your internet’s down, when a provider has an outage, when an API retires a model you depended on, and when a vendor changes its terms. For scheduled work the practice leans on every day, that’s worth something.
What it actually costs
Hardware. A Mac with enough unified memory to run useful models comfortably is a real expense — a good bit more than the base configuration, and you can’t add memory later, so you buy it up front or not at all. On the other side of the ledger: it’s a one-time capital cost, and it’s a machine you were buying anyway.
Setup and maintenance. This is the cost people underestimate. Runtimes update, models get replaced, quantization formats change, something breaks after an OS update. Budget real attention, not a weekend. If nobody in your firm enjoys this stuff, that’s a perfectly good reason not to do it.
Quality, on the hard tasks. Covered above. Don’t pay this cost on work where it matters.
Your time testing it. You won’t know whether a local model is good enough for your extraction task until you measure, which means the evaluation harness from earlier in this section comes first, not later. “It seems fine” is no basis for routing client work.
What local fixes, and what it doesn’t
People sometimes pitch local models as solving the confidentiality problem. They don’t.
Here’s what Rule 1.6(c) requires: “reasonable efforts to prevent the inadvertent or unauthorized disclosure of, or unauthorized access to, information relating to the representation of a client.” Running local shuts one door — sending the data to a provider — and shuts it completely. That’s real, and it’s big.
It does nothing about the rest. The laptop can get stolen. The disk can be unencrypted. The backup can sync somewhere careless. The model’s outputs still land in files that need the same protection as any other work product. A firm running everything locally on an unencrypted machine with no backup discipline is in worse shape on confidentiality than one using a cloud provider under a decent contract with full-disk encryption and a locked screen.
Local is a strong answer to one question. It isn’t an answer to the question.
When I’d tell you not to bother
I think this is most of you:
- If your volume is low. The payoff is concentrated in bulk work. If you process fifteen documents a month, use a cloud provider with good terms and spend the weekend on something else.
- If nobody will maintain it. An unmaintained local pipeline falls apart without telling anybody, and then somebody quietly stops using it. That’s the worst outcome: you paid the setup cost and got none of the benefit.
- If your provider terms are already good. A contract that forbids training on your inputs and limits retention, plus the minimum-necessary habit of sending only what the task requires, is a setup you can defend. It isn’t as strong as never transmitting, and it’s a long way from nothing.
- If you’d use it as an excuse to skip verification. A local model hallucinates just as readily as a remote one. Nothing about the hardware changes the need for the citation gate.
Where to start if you do
- Pick one bounded, high-volume task. Document classification is the natural first candidate — it is mechanical, it runs constantly, and errors are cheap and visible.
- Build the eval set first. Thirty labeled examples. Measure the cloud model’s performance as your baseline.
- Run the local model against the same set. Now you have a number, not an impression.
- Route the task locally only if the numbers justify it — and keep the eval so you notice when a model update changes things.
- Expand one task at a time, in the same order: bounded before open-ended, high-volume before occasional.
Same discipline as everywhere else in this section: decide what you’re measuring, measure it, and let the measurement pick the setup. The confidentiality upside of running local is real enough that you may want it to win. That’s exactly when you should insist on the number.
General commentary on legal technology. Not legal advice, not ethics advice, and not a hardware recommendation. Performance characteristics of specific models change rapidly; nothing above should be treated as current benchmark data. See Minn. R. Prof. Conduct 1.6 and ABA Formal Opinion 512 (July 29, 2024).