Verified Is Not the Same as Defensible

A piece we published recently argued that "verification" in legal AI is doing three different jobs at once: disclosure (did you label the output), confidence scoring (does a judge model's own certainty hold up), and source verification (does the specific claim match a specific real document). The argument was that almost all the energy in the industry goes into the first two, and almost none into the third, because the third is the only one that needs a real domain corpus rather than a general-purpose judge.Sources: Everyone Is Verifying Legal AI (three layers)

The comment section did something a single piece rarely gets to do: it kept arguing after the argument stopped, and it converged on a fourth layer the piece never named. Adel Kildeev opened it: verification does not terminate responsibility, it is one evidentiary step before a qualified professional independently reads the primary sources and takes ownership of the decision. Marco Rossi named it in three letters, LITL, lawyer in the loop. Itamar Rosen sharpened the question further: should a pilot announce every autopilot engagement to the passengers, or does the passenger actually need something else, a trained, accountable pilot and a black box that reconstructs what happened if something goes wrong.

Then Arkadiy Miteiko, replying directly under the piece, compressed the whole idea into one line: "computational correctness is not institutional validity." The point underneath it is that a correctly sourced claim can still fail depending on who is relying on it, under what authority, for what matter, at what moment, and that trust in legal AI comes less from citations existing and more from being able to reconstruct and justify every consequential use after the fact.

That is not a restatement of source verification with different words. It is a fourth, separate axis, and it has a name and a body of work behind it: Ignacio Adrian Lerer, a corporate lawyer and independent director who writes on exactly this boundary between a system's output and an institution's ability to stand behind it.

The boundary Lerer keeps drawing

In a recent piece on Argentina's pending reform around autonomous corporate decision systems, part of the debate around introducing an autonomous corporate form into the Ley General de Sociedades, Lerer reframes the whole question. He is less interested in whether a system can hold legal personhood than in a narrower, harder problem: when an autonomous system does something nobody explicitly signed off on, whose decision was it. His answer separates two roles that get treated as one and shouldn't be. Whoever set up the system, chose its parameters, and defined what it was authorized to do is answering for a different act than whoever, or whatever, acted autonomously inside that setup. Treat them as the same actor and you either let the setup decision hide behind the autonomous one, or you hold someone accountable for behavior nobody could have configured against in advance. He grounds the proposal deliberately in Argentina's constitutional article 19: regulate what happens when the system causes harm, not the internal steps by which it decided anything. Control of outcomes, not control of process.Sources: Ignacio Adrián Lerer, on the SCBA AI framework (Abogados.com.ar) · Argentina's "sociedad automatizada" reform (Infobae)

That distinction, configurator versus executor, is the cleanest available frame for what "defensible" actually means. A verified claim tells you the executor produced something that matches a real source. It tells you nothing about whether that specific act of reliance happened inside the boundary the configurator actually authorized, at that moment, for that matter, under that mandate. Those are different questions, and only one of them is answered by checking the citation.

Where it gets concrete: the expert witness who cannot explain the reasoning

Lerer's other relevant piece, on a judicial AI framework the Supreme Court of Buenos Aires Province just approved, makes the same distinction concrete rather than theoretical. The regulation requires "significant" human supervision of AI-assisted judicial work without ever defining what that means in practice, an open threshold sitting inside a binding rule. The example he gives cuts through the ambiguity on its own: a court-appointed expert who signs off on an AI-drafted report without genuinely following its reasoning has a real problem the moment opposing counsel starts asking questions on the stand, a problem that has nothing to do with whether the citations in that report check out. What actually gets tested on cross-examination is the adversarial guarantee, the expert's own ability to defend their conclusions against a hostile question, not the accuracy of what the report cited. The same regulation, in the same breath, makes checking an AI-generated citation mandatory for judicial staff. Both requirements sit side by side in one rule. Only one of them can be satisfied by running a check.Sources: SCBA AI framework, Resolución SC 1.719/26 (Diario Judicial) · SCBA official note

This is the pattern worth naming precisely: a correctly sourced claim handed to someone who cannot reconstruct why they relied on it is still a liability. A weaker, unverified claim that the person fully understands, can explain, and takes ownership of might survive scrutiny the sourced one does not, because scrutiny in an adversarial proceeding tests the human's authority and understanding, not the citation's existence.

What defensibility actually requires, mechanically

If source verification needs a real corpus, a way to bind a specific claim to a specific document, defensibility needs a different, harder set of things.

Authority bound to a moment, not just a role. Standing to rely on an AI-assisted output is time-specific and mandate-specific, not a permanent property of a job title. Verifying that a citation is real says nothing about whether the person invoking it currently holds the authority their reliance implies, for this matter, at this stage of it. Lerer's configurator-executor split makes the failure mode precise: the question is never just "did the system act correctly," it is "was this specific act of reliance inside the scope the configuration actually authorized." A signature valid before a mandate lapses is not automatically valid after. A tool authorized for one category of matter does not carry authorization into an adjacent one just because nobody re-checked.

Reasoning preserved, not just conclusions approved. A system that hands a professional a final answer with citations attached lets that person approve a conclusion without absorbing the reasoning that produced it. Cross-examination does not test conclusions. It tests reasoning, under pressure, in real time, from someone whose job is to find the seam. A workflow built for defensibility has to force the human to walk through and affirm the reasoning chain itself as a distinct step from accepting the final citation-checked output, because those are not the same act of review, and only one of them produces someone who can survive being questioned about it.

The record has to name the accountable actor, not just log that a check ran. A flight recorder that shows autopilot engaged at a given moment but not which pilot was in command and accountable at that moment is missing the part that matters when something goes wrong. The equivalent failure in legal AI is a system that logs "verification passed" without binding that pass to a specific person who reviewed it, under a specific authority, on a specific date, and can be asked about that decision later. Verification produces a fact about a claim. Defensibility requires a fact about a person's relationship to that claim, at that time.

Undefined oversight standards are a live gap, not a hypothetical one. Lerer's point about "significant human supervision" is not an abstract critique of regulatory drafting. It is a description of a rule that exists right now, binding, with a standard nobody has operationalized. A rubber-stamp review, five seconds of clicking approve, technically satisfies "a human supervised this" under a rule that never says how much supervision counts, while providing none of the defensibility the rule exists to produce. Any system claiming "human in the loop" as a control needs to be able to say, concretely, what distinguishes supervision that would hold up from supervision that would not, because the rule usually does not say, and the gap defaults to whoever is asking the question after something has already gone wrong.

Why this does not collapse into verification

It would be tidy if defensibility turned out to be source verification wearing a different name, one more sub-check to add to the same pipeline. It is not, and treating it that way loses the actual point. Source verification answers a question a regulator, an opponent, or an auditor asks about the content: does this claim match reality. Defensibility answers a question a court, a tribunal, or a board asks about the person: did you have the standing and the understanding to rely on this, here, now, and can you show your work. A system can produce a perfectly verified claim and still fail the second question completely, and a system can produce something weaker that fully survives it, because the second question was never about the claim in the first place.

That is also why the fix does not look like a scoring pipeline. Verification can be automated end to end, because it is a property of a document matched against another document. Defensibility cannot be automated the same way, because it is a property of a specific human's relationship to a specific decision, and the most that architecture can do is make that relationship reconstructable rather than assumed. What it can guarantee is that the reasoning was actually walked through rather than rubber-stamped, that the record names who decided what under which authority, and that the gaps in what counts as adequate oversight are treated as open questions to be pinned down rather than papered over with a checkbox.

None of this was planned as a second piece when the first one went up. It came out of an actual comment thread, in real time, from people who pushed the argument further than the original draft did. That is a strange way to build an argument, in public, with the people who are going to use it, but it produced a sharper distinction than a closed room would have, and the honest thing is to say so.