Twelve Questions to Ask a Legal AI Vendor, and the Answers That Should Worry You

Most guidance on choosing legal AI is written by people selling it, which is why it tends to list features rather than questions. Features are easy to answer and hard to check. A question with a wrong answer is neither.

What follows is the list we would use, and it is not a comfortable list for us either. Several of these are questions we could not have answered well a year ago, and one of them describes a defect we found in our own system and had to take apart. They are ordered by how much damage a bad answer does.

Ask what the output says when nothing was checked

The first question is the one almost nobody asks: when your system cannot verify something, what exactly does the result say? A verification step that reports pass or fail has one bit to work with, and at least four states behind it: checked and clean, checked and broken, nothing to check, and could not check. If the last two collapse into the first, a clean result and a silent result are indistinguishable to the person reading them, and no amount of care downstream recovers the difference. A good answer names an explicit third state. A bad answer is a demonstration where everything passes.

Then ask whether an unusually clean result triggers anything. A document with fewer things to check than that class of document normally contains is a signal in its own right. If the system treats it as reassuring rather than as a prompt to ask why, the quietest failures are the ones it will never surface.

And ask what the check actually ran over. A result is only interpretable next to an honest statement of the working set: how many items were in scope, and what they were. A vendor who reports the result but not the population is reporting half a measurement and presenting it as a whole one.

Ask where the ground truth comes from

Ask who produced the evidence the verification runs against. If the same party that produced the claim also produced the material it is checked against, what you have is a consistency check wearing an authenticity check's clothes. It will be internally coherent, every part agreeing with every other part, and it will never have been asked to agree with anything outside itself. This is the defect we found in our own audit trail, and it took two colleagues asking the uncomfortable version of the question out loud, then testing it against the real system rather than arguing about it.

Ask what happens to a provision that has been repealed. In a codified system the paradigm failure is not a fabricated citation. It is a real article, quoted correctly, that stopped being in force and is still being cited as current law. If the vendor treats repeal as a missing record or an error rather than as a verdict in its own right, the system is checking whether text exists, not whether it governs.Sources: why a repealed law is not a stale fact

Ask what the timestamp on a source actually means. In most retrieval systems it records when someone last touched the record, not whether what it says is still true. Those are different questions with different answers, and the gap between them is where confidently wrong output lives.

Ask about the jurisdictions you actually work in

Ask for the difference between a jurisdiction that is indexed, one whose corpus has merely been collected, and one the model can only reason about from general knowledge. All three can look identical in a demo. Only the first gives you retrieval against real sources. A vendor who will not draw that line for you is one whose coverage number is doing work it cannot support.

Ask what the system does with a jurisdiction it has not modelled. There are two possible answers and only one of them is safe. It can decline, or it can produce something by applying a neighbouring jurisdiction's rules. For anything with a deadline attached, the second is the worse outcome: a plausible date gets acted on, the filing is late, and there is nothing left to fix. Ask to see the refusal.

Ask whether the verification logic was built for the language it will run in, or translated into it. The ways a legal claim goes subtly wrong, a negation flipped, a quantifier weakened, an exception dropped, are language-specific patterns. A system adversarially tested in one language has no guarantee of catching the equivalent failure in another, and the failure mode is silence, not an error.Sources: verification does not travel with translation

Ask what happens when it is wrong

Ask from whose professional standpoint the analysis is written. The same contract and the same law produce different findings depending on whether the question is whether a deed can be authorised, whether an entry is registrable, or when to file. A defect that is irrelevant to a litigator can be the finding for a registrar. If the output does not state the standpoint it was written from, it has quietly picked one for you.

Ask what a single flaw costs across a portfolio. A tired professional produces errors that scatter; a model with a flaw produces the same error every time the pattern recurs, across every matter, at once. This is not a claim about which is more accurate on average. It is that uncorrelated error averages out across many decisions while correlated error compounds, and no aggregate comfort helps the file it landed on.Sources: the asymmetry of trust

Finally, ask what the record looks like to somebody who was not there. Not whether there is a log, everything logs. Ask whether the record shows what was checked, what was not, what the check ran over, and who is answerable, in a form a regulator or an opposing party could evaluate without taking the vendor's word for any of it. That is the difference between a system that is auditable and a system that has an audit trail.

How to read the answers

None of these questions has a single correct answer, and any vendor who answers all twelve without hesitating is either exceptional or has heard them before and prepared. What matters is the shape of the answer. Specific beats fluent. A named limitation beats a reassurance. An offer to show you the failure case beats a demonstration in which everything works.

The reason to ask them before signing rather than after is that most of these defects are invisible from the outside precisely when they are doing damage. A system that only ever reports problems it found has, by construction, nothing to say about the problems it never looked for.