Assurance Without an Accreditor: Building Verification Infrastructure Before Anyone Can Check Your Work

On July 7, 2026, the European Commission published its Action Plan on Cybersecurity and Artificial Intelligence (COM(2026) 577 final). Buried in Section 2.2 is a commitment to stand up a European evaluation capacity for frontier AI models: an external body that will assess what models can do, whether provider safety mitigations actually hold, and support compliance with the AI Act and the GPAI Code of Practice. It is to be established through a dedicated call, with the capacity targeted to become operational in 2027. The Action Plan promises that this capacity will ensure "a credible and rigorous external evaluation process."Sources: COM(2026) 577 final (EUR-Lex)

Attached to that sentence is footnote 20. It states that this capacity will not be a conformity assessment body, and will only support compliance through its evaluations rather than certify it directly.

Read narrowly, the footnote is a scoping clause. Read as a pattern, it describes something much more common than a single EU policy document: an institution being asked to produce assurance before the apparatus that would make that assurance independently checkable exists. The EU's own accreditation infrastructure has nothing on the shelf for evaluating frontier model capabilities. The relevant management-system standard, ISO/IEC 42006, certifies bodies that audit AI governance processes. It says nothing about bodies that test what a model can actually do. Evaluation science for frontier models is a few years old. Forcing it through an accreditation cycle designed for mature, stable methodologies would freeze it before it has had the chance to mature.

So the decision to launch without accreditation is defensible. What is not defensible, and what the EU document does not resolve, is what happens as the results produced by this informal capacity get used for increasingly consequential decisions, while the capacity itself remains exactly as informal as the day it launched. Every report that lands on a regulator's desk with this evaluation attached makes the evaluation more load-bearing. Nothing in the informal structure adjusts in response. An institution can be perfectly legitimate at launch and structurally unaccountable five years later, without a single decision along the way that looks wrong in isolation.

This is not a problem specific to one EU policy instrument. It is the condition of almost every new verification or evaluation function in a fast-moving field, including the one we are building.

The general shape of the problem

Strip away the EU-specific detail and the pattern is this: a new kind of claim needs checking (does this frontier model actually behave safely, does this legal citation actually reflect current law, does this AI-generated compliance record actually hold up), and no external accreditor yet exists with a methodology mature enough to certify anyone who checks it. Two options present themselves, and both are wrong in a specific, instructive way.

The first wrong option is to wait. Refuse to publish an evaluation, a verification, a rating, until an accreditor exists to bless the methodology. This sounds responsible and is actually an abdication, because the accreditor that does not yet exist cannot come into existence without a body of evaluation practice to study, criticize, and eventually standardize. Accreditation regimes are built by watching real evaluators make real judgment calls, including bad ones, over years. Refusing to evaluate anything until that process has already happened elsewhere just means someone else's informal evaluation practice becomes the de facto standard while yours stays theoretical.

The second wrong option, and the more common one, is to evaluate anyway and let the output look exactly as authoritative as if an accreditor stood behind it. A report with a score, a named methodology, a clean pass or fail, reads the same to a downstream decision-maker whether it came from a battle-tested accreditation regime or from a team that invented its rubric six months ago. Nothing about the visual and rhetorical form of an evaluation report signals how much epistemic weight it can actually bear. This is the failure mode that matters, because it is invisible from the outside and comfortable from the inside. Nobody has to lie. The evaluator can be completely sincere and still produce something that gets relied upon far past what its own process can support, simply because the report doesn't distinguish between what was actually stress-tested and what was a reasonable best guess under real constraints.

The honest third option, the one this article is about, is to evaluate now, refuse the appearance of an authority you do not have, and build the specific disciplines that let your own process be audited by anyone, at any time, without waiting for an accreditor to do it for them.

Why the single confidence number is where trust becomes unwarranted

The mechanism by which informal evaluation quietly becomes over-relied upon almost always runs through the same design choice: collapsing two structurally different epistemic states into one output.

State one is a claim that has been checked against something external and verifiable: a citation resolved to a real source that says what it's claimed to say and is still in force, a model behavior reproduced under controlled conditions against a documented test, a compliance record cross-checked against a ground-truth log. State two is a judgment call made under genuine uncertainty: an interpretation where reasonable experts would disagree, a case where the underlying methodology hasn't yet been validated against enough examples to know its error rate, an edge case that the evaluator flagged internally but resolved with a best guess rather than a settled answer.

Both states are legitimate parts of doing real evaluation work under time and resource constraints. The failure is publishing them identically. A single score, a single pass or fail, a single confidence percentage, erases the distinction a downstream reader would need to calibrate how much weight to put on the result. And because state two work often looks and reads exactly like state one work, once it is expressed in the same format, the discipline required to keep it honest disappears at the exact moment it is needed most: under pressure to look decisive, to close out a review, to give a client or a regulator a clean answer instead of a qualified one.

This is not a hypothetical failure mode. It is the default behavior of most evaluation and rating functions under commercial or political pressure, because a report that says "confirmed" and "genuinely unresolved, here is why" in two visibly different sections reads as less authoritative than one that reports a single tidy number, even though the first is the more honest and, over time, the more trustworthy of the two.

The disciplines that substitute for an accreditor you don't have yet

If accreditation is not available, the alternative is not less rigor. It is rigor that does the accreditor's job on yourself, made visible enough that anyone external can perform the check the accreditor would have performed, without needing the accreditor's institutional standing to do it.

Separate confirmed from unresolved as a structural output, not a footnote. Every evaluation result should carry an explicit, first-class status alongside its content: this was checked against an external, verifiable ground truth and passed; this was a judgment call under genuine ambiguity and resolved this way for these stated reasons; this is a known gap where the methodology does not yet have coverage. These are not gradations of the same score. They are different kinds of claims, and collapsing them into one number is precisely the design choice that lets informal evaluation quietly outgrow what it can actually support. A reader who can filter a report down to only the confirmed claims has a genuinely different, more defensible document than one who has to trust the aggregate.

Publish the methodology as a versioned artifact, not a one-time description. An accreditor's real function is less about blessing a single evaluation than about tracking how an evaluator's standards change over time and catching drift. In the absence of an accreditor, that tracking has to be self-imposed and public: every material change to how evaluations are performed gets a version number, a changelog entry, and a date, so that anyone auditing a result from eighteen months ago can see exactly which methodology produced it and how that methodology has since been revised. Static, undated methodology documents let an evaluator's standards quietly erode under commercial pressure without anyone being able to prove it happened.

Keep the claim narrow enough to be falsifiable. The broader a claim, the less anyone can check it. "This AI system is trustworthy" cannot be verified or refuted by an outside party; it is a halo statement dressed as an evaluation. "This specific output's citation resolves to this specific source, which was in force as of this date, verified against this specific version of the underlying register" can be checked by anyone with access to the register, independent of any trust in the evaluator's institutional standing. Every evaluation capacity operating without accreditation should be narrowing its claims until each one is independently checkable by a party with no reason to trust the evaluator at all. Breadth of claim is where unaccountable authority hides.

Adopt adversarial self-testing as a standing practice, not a one-time audit. Since no external accreditor is available to try to break your methodology, that role has to be filled internally, and filled by people whose incentive is to find the failure, not to defend the process. This means deliberately seeking out the cases most likely to embarrass the evaluation, publishing the failure rate alongside the success rate, and treating a disconfirming case as more valuable output than a confirming one. An evaluation capacity that only ever reports its successes is not doing the accreditor's job on itself; it is doing marketing.

Name, in advance and in public, the condition that ends the informal period. The single clearest lesson from the EU's own footnote 20 is that informality left undated becomes permanent by default, not by decision. The corrective is to state explicitly, before external pressure forces the question, what would need to be true, what volume of validated cases, what level of methodological maturity, what independent replication, before this evaluation capacity actively pursues formal accreditation or external validation rather than continuing to operate informally by default. Setting that condition publicly is what prevents an informal capacity from becoming a permanent, unaccountable one simply because nobody with the authority to formalize it ever had to answer for not doing so.

What this does not solve

None of the above is a substitute for real accreditation once one becomes available. Self-administered rigor has an irreducible limit: the evaluator is still grading its own homework, and even a scrupulously honest evaluator cannot fully replace the function an independent party serves, precisely because independence is the one property self-administration cannot produce. The disciplines above are a bridge, not a destination. Treating them as sufficient forever is exactly the failure Patrick Sullivan, VP of Strategy and Innovation at A-LIGN and a member of ISO/IEC JTC1/SC42, warns about in his reading of footnote 20 to COM(2026) 577 final, applied one level down: an evaluator that builds excellent internal rigor and then stops pushing for external accreditation once the internal version feels adequate has recreated the same informal-but-load-bearing trap the EU document risks, just with better internal engineering.

The test of whether a bridging discipline is honest is whether it is actively working to make itself obsolete: whether it is tracking, publicly, the conditions under which it would hand its function to a real accreditor and step back. An evaluation capacity that has quietly stopped wanting that day to arrive has stopped being a bridge and become exactly the informal-but-load-bearing institution Sullivan warns against, only with better documentation.