An organization can accurately report that a person is in the loop and still operate a decision path into which no person ever looks.
Both statements are true at once because they are about different things. “There exists a reviewed route” is not “every consequential route is reviewed.” The label is not false so much as underspecified at exactly the level where the safeguard varies — and it invites an inference the architecture does not support.
If you audit AI systems, that gap is what you are looking for. This article gives it a name, shows how to measure it, and works it through two intake workflows: hiring and clinical-trial pre-screening.
The pattern has a shape
Take an AI system that screens job applications. Applications scoring above a threshold reach a recruiter. Applications below it are rejected without recruiter review. A person genuinely participates in the workflow. They just do not participate in every consequential branch.
Call this asymmetric human oversight: the configured assignment, timing, depth, or authority of meaningful operational review depends on the system’s output, or on the route that output creates.
“Asymmetric” means non-uniform, not binary. A three-tier triage system that clears low-risk cases automatically, routes medium-risk cases to a nurse, and requires physician review for high-risk cases is asymmetric across three routes. The unit that matters is the operational partition that routes actions — not the binary vocabulary a vendor happened to choose.
The definition is deliberately neutral about whether the asymmetry is justified. It describes an architecture, not a violation. The normative concern is strongest where a branch combines consequential decisions, low review coverage, poor verification, limited contestability, and unequal error costs. Naming the pattern makes that combination visible without assuming that uniform review is the remedy.
Human presence does not propagate
Whether a route is human-gated has to be stated for a specific action at a specific point of effect.
A model-generated ranking of applicants can take effect without review even though a person makes the later hiring decision. Conversely, a recommendation held in abeyance until someone authorizes it is not autonomous with respect to that action.
A route is operationally autonomous relative to an action when the model output is sufficient, under the configured rules, for that action to take effect without further case-specific human authorization.
What does not make a branch human-gated: that humans designed the policy, chose the model, set the threshold, and remain accountable for the system. Nor batch monitoring, an on-call reviewer, random verification, or a later appeal. Those are real controls — they can make an autonomous branch monitored, testable, contestable, or reversible. What they are not is prior review of the individual case, and describing them as such overstates the safeguard.
A useful reporting vocabulary falls out of this:
| Execution pattern | Condition | Honest description |
|---|---|---|
| Advisory or deferred | Model output alone cannot make the action take effect; a downstream person decides or authorizes it | ”AI-assisted; the specified action is human-decided” |
| Human-gated | Configured rules prohibit every proposed action from taking effect without meaningful review holding preventive authority | ”Mandatory pre-effect review for this action,” with reviewer role and authority stated |
| Selectively human-gated | The pre-effect requirement applies to only some proposed actions | State the selection rule and the percentage gated. Do not describe the whole route as human-reviewed |
| Operationally autonomous | Configured rules permit every proposed action on the route to take effect without case-level review | State that the action can take effect without case-level review; report post-review, verification, and contestation separately |
“Operationally autonomous” describes execution at a decision node. It does not imply machine agency, does not erase organizational responsibility, and does not by itself establish a solely automated decision under GDPR Article 22.
Why this happens, and what it costs
Asymmetric oversight usually arises for a defensible reason. Human attention is costly and slow. The model is introduced to filter volume, direct specialists to uncertain cases, enforce consistency, or make an appeal possible without re-reviewing every initial decision. This is not an implementation accident. It is often the business case.
The price is that review allocation also allocates organizational knowledge.
The reviewed branch generates corrections, reasons, disagreements, and often labels. The other branch produces throughput. When decision logs are later turned into a performance dashboard or a training set, that difference disappears behind rows of apparently complete data. The table contains a model output for every case. It does not contain an independently established truth for every case.
Which error goes silent depends on which side is unreviewed, and the direction is not fixed:
- Gatekeeping asymmetry — screen-ins reviewed, screen-outs not. False negatives disappear from the routine review stream.
- Exception asymmetry — alerts investigated, ordinary cases passed. Misses on the quiet branch go unexamined.
- Appeal-triggered asymmetry — an adverse action reaches a person only if someone contests it.
- Clearance asymmetry — failures and no-matches get manual fallback while successful matches auto-clear.
- Confidence asymmetry — uncertain cases deferred, confident predictions on both sides automated.
These are overlapping lenses rather than disjoint categories, and one deployment can combine several. The same architecture can hide under-inclusion or under-enforcement.
Two coverages, not one
The single most useful move in an audit is to stop treating human review and independent verification as the same control.
- Case-review coverage asks whether a person meaningfully reviewed this case.
- Reference-verification coverage asks whether an independent determination can test the system’s output.
They come apart in all four combinations:
| Verification present | Verification absent | |
|---|---|---|
| Reviewed | Reviewed and testable. Remaining question: is the review independent and meaningful? | Reviewed but epistemically weak. A person acts, but model error may never be established. |
| Not reviewed | Automated path with an evidence channel — random adjudication, or a downstream outcome valid for the stated target. | Operationally unreviewed and unverified. |
The bottom-right cell is the danger, and it earns a name. A consequential route is a blind branch when three things hold together:
- The action takes effect without meaningful pre-effect review.
- The route lacks evidence adequate to assess the relevant error.
- No practicable contest mechanism exists.
Practicability is part of the definition. An appeal against a rejection the affected person was never notified of does not count.
Note the bottom-left cell too, because it cuts against the intuitive fix. Independent random verification of an automated branch can give you stronger system-level evidence than nominal review attached to every case. Asymmetry is not automatically a monitoring failure, and universal review is not automatically the answer.
Verification changes the evidentiary status of a branch, not its mode of execution. An autonomously executed action can be extensively verified after the fact, but retrospective verification does not provide preventive case-level oversight, and it does not retroactively make the initial action human-gated.
Two workflows
Both workflows below are intake filters with gatekeeping asymmetry. They look structurally identical on a slide. They differ sharply in one respect that determines what evidence is even obtainable: whether the thing you would need to verify still exists after the decision.
Example 1 — Hiring: the target is destroyed by the decision
A rejected application can be re-read. Performance in a job the applicant never held cannot be recovered from the record.
The workflow. An applicant tracking system scores incoming applications. Applications above threshold go to a recruiter, then an interview panel, then a hiring manager. Applications below threshold receive an automated rejection.
The branches.
| Screened in | Screened out | |
|---|---|---|
| Case review | Recruiter, panel, and manager review | None. Exits before a person assesses the full application |
| Execution | Advisory — a person decides the hire | Operationally autonomous for the rejection |
| Reference verification | Interview outcomes, but anchored on the same application | Typically none |
| Outcome linkage | Job performance for those hired | Structurally unavailable |
What the reviewed branch can establish. Positive predictive value of a sort — how many advanced applicants the panel judged worth hiring. Routing volume, review completion, latency, and recruiter disagreement with the score. All useful. None of it is a full account of model performance.
What goes silent. A qualified applicant is denied an opportunity, and the exclusion may never appear in outcome data. The applicant typically receives a generic rejection and cannot see that a parser misread their work history. Automated screening can also screen out applicants with disabilities who are able to perform the job.
The hard part, and what makes hiring distinctive. Re-reading a rejected application can establish that stated screening criteria were applied incorrectly. That is a real, auditable finding, and it is worth sampling for.
What re-reading cannot do is reveal how that applicant would have performed in the denied job. That outcome does not exist. The decision destroyed it. This is selective labels: the rejected applicant is not hired, so their performance in the denied role is counterfactual. No log recovers it. Carefully designed exploration, natural experiments, or explicit causal assumptions may support estimation; ordinary operational records do not.
So the audit has to split the question:
- Was the stated rule applied correctly? — Verifiable. Sample the screen-out branch and re-adjudicate against the documented criteria.
- Was the rule itself any good at predicting performance? — Not identifiable from routine data. Any claim here rests on assumptions or on a causal design, and should say so.
An organization that conflates these two will report the first and imply the second.
Example 2 — Clinical-trial pre-screening: the target survives the decision
With a sufficiently complete record, protocol eligibility can be independently assessed even after a person is screened out.
The workflow. A model pre-screens potential trial participants against protocol criteria. Those predicted potentially eligible pass to clinical research staff and, where appropriate, to an investigator for formal assessment. Those predicted unlikely to be eligible do not pass on.
The branches.
| Advanced | Screened out | |
|---|---|---|
| Case review | Clinical research staff, then investigator assessment | None before exit |
| Execution | Advisory — a person adjudicates eligibility | Operationally autonomous for the exclusion |
| Reference verification | The operational assessment itself — but see the independence caveat | Feasible, and this is the key difference |
| Outcome linkage | Enrollment, and trial outcomes for those enrolled | Not needed for the eligibility target |
Why this case is structurally better than hiring. Eligibility under a trial protocol can, in principle, be adjudicated from a sufficiently complete clinical record independently of whether the person was advanced. The target still exists after the decision. Unlike denied job performance, nothing was destroyed. That makes branch-stratified verification of the screen-out branch genuinely available — which means a blind branch here is a choice, not a constraint.
Note also that the same branch can be verifiable for one target and not another: verifiable for protocol eligibility at screening, and unverifiable for therapeutic benefit years later. State the target and the horizon, always.
The independence caveat. The advanced branch’s “verification” is often the operational eligibility assessment itself. An investigator who sees an incomplete record plus the model’s rationale may reproduce the model’s error. Verification has to be sufficiently independent for the performance claim being made — ideally adjudicated without anchoring on the model output.
What the numbers do. Here is the illustration from the paper. Treat it as arithmetic, not as evidence about any real deployment.
A workflow screens 10,000 potential participants. The model advances 800; independent adjudication of all 800 finds 520 eligible, so PPV is 65%. From the 9,200 screened out, a blinded random sample of 300 finds six who were in fact eligible. The branch false omission rate is 6/300 = 2%.
Weighting back to the branch gives roughly 184 estimated false negatives against 9,016 true negatives. And now the reconstruction:
| Quantity | Point estimate | Approx. 95% interval |
|---|---|---|
| False omission rate (screen-out branch) | 2.0% | 0.92–4.29% |
| Negative predictive value | 98.0% | 95.71–99.08% |
| Specificity | 97.0% | 96.92–97.02% |
| Overall accuracy | 95.4% | 93.25–96.35% |
| False-negative rate (system) | 26.1% | 14.0–43.2% |
A dashboard for this screener can truthfully report a 2% false omission rate, 98% NPV, and 95.4% accuracy while the workflow misses an estimated 26.1% of all eligible participants — with the evidence compatible with anywhere between roughly one in seven and three in seven.
Two things are worth sitting with. First, the specificity interval of 96.92–97.02% next to 14.0–43.2% for the miss rate looks like a misprint. It is not: the true-negative count dominates specificity’s numerator and denominator and barely moves, while the same 300-case sample leaves the miss rate wide open. A precise-looking headline metric is not evidence that the metric anyone cares about is precise.
Second, the two rates that get conflated are genuinely different questions. The false omission rate is the share of screened-out cases that were actually eligible — what a verification sample from that branch directly estimates. The false-negative rate is the share of all eligible cases the system failed to advance — which additionally requires knowing the eligible population across both branches. In low-prevalence screening, true negatives usually outnumber true positives, so labeling a sample’s false omission rate as the false-negative rate will commonly understate the miss rate.
The cost that runs the other way. Adjudicating a screened-out branch means retaining and re-examining clinical records of people who never consented to that trial. That cuts against storage limitation and purpose limitation, and declaring verification necessary does not resolve it. The answer is the least intrusive design that can support the claim: a bounded sample sized to a stated precision target rather than standing access to the excluded population, pseudonymized or aggregated adjudication where the reference standard permits, retention limited to the evaluation period, and a documented lawful basis for the secondary use. Where no such design exists, the honest conclusion is that the branch cannot support the performance claim — which argues for stronger prevention or a narrower action, not for making the claim anyway.
What to do about it
Four numbers to ask for
For each consequential route and action, ask for these separately. Their divergence is the finding.
| Quantity | The question it answers |
|---|---|
| Case-review coverage | What share of cases on this route got review meeting a pre-specified standard? |
| Pre-effect gate coverage | What share of proposed actions are configured to require review before taking effect? |
| Reference-verification coverage | What share can be tested against a sufficiently independent determination? |
| Outcome-linkage coverage | What share have a relevant downstream outcome linked back? |
Three things make these answerable rather than rhetorical.
A minimum review standard, assessable from records. Who is competent to review; what evidence and model information they see; timing relative to the action’s first practical effect; minimum depth, time, or procedural steps appropriate to the risk; and the authority to prevent, modify, or escalate. If fulfillment cannot be checked from inputs shown, timestamps, reviewer role, available actions, reasons, and disposition, the coverage number means little.
Configured target reported alongside realized coverage. This separates designed allocation from staffing failure and drift. Identical policies produce different realized coverage; equal realized percentages can conceal different gates. A branch designed for 10% review can quietly fall to 2%.
Resistance to the one-number dashboard. A single summary statistic — the spread between best- and worst-covered branch, say — is blind to direction, case volume, and consequence severity. It discards the distribution once there are more than two branches. And it can be improved by lowering review on the better-covered branch. Keep the branch table, paired with action counts and a documented assessment of severity and reversibility.
Coverage should also be stratified jointly with performance — by protected group, score band, site, model version, and time. If errors, missing data, or appeal access differ across groups, the unverified branch can hold disproportionate numbers from already disadvantaged ones. Because those cases supply no ground truth, a fairness dashboard computed on the reviewed sample can show parity while the deployed system remains unequal.
Meeting the standard is not the same as working
A branch reviewed in 100% of cases can still perform badly if reviewers see only the recommendation, work under throughput pressure, or hold formal authority they cannot realistically exercise. Automation bias and complacency are well documented; people may also adhere selectively when advice aligns with stereotypes; and field work shows humans do sometimes catch erroneous scores and depart from them. The evidence does not justify a universal presumption in either direction.
Which means an override rate alone establishes nothing. A low rate may indicate deference, an accurate model, a narrow review mandate, or a threshold that only sends obvious cases. A high rate may indicate useful correction, poor model quality, or deliberate use of the model merely to prioritize. Better measures compare independently adjudicated accuracy before and after review, record whether reviewers correct known model errors, examine disagreement by case difficulty and subgroup, and assess whether reviewers had the information and authority required to act.
Coverage is extent. Effectiveness is a separate empirical property of a designed interaction, and needs its own evaluation.
Sampling an unreviewed branch
Representative verification can reveal branch errors without making every action human-gated.
One rule is worth carrying in your head. For an equal-probability, approximately independent sample with a sufficiently accurate reference standard, if n adjudicated cases from an unreviewed branch turn up zero errors, the approximate one-sided 95% upper bound on that branch’s error proportion is 3/n.
Two qualifications do the real work. It bounds the error rate in that branch — not the system-wide false-negative rate, which requires combining uncertainty across several estimated components. And it does not hold unmodified for stratified, clustered, adaptive, or unequal-probability designs, repeated people, or sampling a large fraction of a finite population. Those need a design-specific bound.
Random adjudication is not the only route. Depending on the domain: full verification where consequence is severe or volume low; delayed linkage to independent outcomes; shadow deployment before the model is allowed to gate anything; near-threshold randomization or regression discontinuity where a routing threshold already exists and cases either side are otherwise comparable; independent second-reader or panel adjudication.
Appeals deserve special caution. They are selected by awareness, persistence, literacy, time, trust, and access — so the people least able to contest a decision are the least represented in appeal data. An overturn rate can reveal a problem. A low appeal rate cannot establish that initial decisions were correct. Appeal review protects individual contestability; sampled verification estimates under-enforcement. Different problems.
And a caution on correction methods: if verification probability is structurally zero for a branch, non-parametric identification from routine data fails. Parametric, Bayesian, or causal models can extrapolate into that branch, but they do so through assumptions rather than newly observed evidence. The practical lesson is that the evidence-generating design must precede the dashboard.
Proportionality: when asymmetry is defensible
Symmetry is not an end in itself. Universal review can invade privacy, delay beneficial actions, and waste expertise that belongs on higher-risk cases.
Three controls do different work and are not substitutes:
- Pre-effect review can prevent an individual action.
- Representative verification can estimate performance across a class of actions.
- Contestability can remedy a particular decision and supply a sentinel signal.
A random verification sample does not protect a person from an irreversible action. An appeal sample does not identify a population error rate.
That suggests a severity-scaled evidence floor, applied per route–action pair rather than per system:
| Consequence | Floor |
|---|---|
| Severe or irreversible | Pre-effect gating for the consequential action, plus evidence adequate to the relevant error. Monitoring quality alone does not discharge this — retrospective verification can correct a policy but cannot reopen a closed opportunity. Rebuttable, but departure needs a documented showing that pre-effect review would itself increase expected harm, with compensating controls stated. |
| Moderate and reversible | Branch-stratified verification sized to a pre-specified claim, precision, and detection time, plus notice and a reachable reviewer with authority to change the outcome. Case review may be sampled. |
| Low and readily reversible | Sentinel signals plus accessible redress, with an explicit statement that this supports detection of gross failure only and cannot support a population error-rate claim. |
Two constraints keep this honest. Severity should be assessed from the affected person’s position, not the organization’s: a screen-out costing an organization one marginal candidate may cost that person a job or access to an investigational treatment. And the assigned tier should be recorded with its rationale and revisited when volume, population, or threshold changes — the least demanding tier is always locally attractive, and an unexamined ladder becomes a way of ratifying whatever evidence you already happen to hold.
Escaping the blind-branch definition is a floor, not a target. One control removes a route from that category without supplying the other two functions.
Say it in a way that can be checked
The alternative to an unqualified label is branch-qualified disclosure. Something like:
This workflow uses asymmetric oversight and heterogeneous execution. Cases routed to [Branch A] are assigned to meaningful human review before [action] in [configured coverage] of cases; realized coverage was [observed] during [period]. Reviewers see [information] and can [authority]. On [Branch B], [action] can take effect without routine case-level review and is operationally autonomous at that point. [Branch B] receives [verification coverage, target, method, horizon]. Affected persons receive [notice] and may obtain [contest mechanism]; filing a challenge [does / does not] suspend the initial action. This statement covers [model version and population].
Where percentages vary, report configured targets and observed ranges rather than converting partial coverage into a categorical claim. And keep individual oversight separate from system monitoring: “5% are independently adjudicated each month” is an evidence statement, not proof that the other 95% were human-reviewed.
Less elegant than “the system is human-in-the-loop.” Also testable.
A note on the legal frame
The EU AI Act’s human-oversight provisions and GDPR Article 22 both bear on this territory, and for Article 22 purposes the branch can be the decision — a point already raised in the literature on selective automation in multi-stage profiling.
But neither instrument prescribes branch-level coverage metrics, and nothing here should be presented as a compliance requirement. The framework is a way to make an oversight claim auditable, which is a different and more modest thing than a legal obligation. The tiers above are a proposed default and a professional recommendation, not a legal standard.
The question that replaces the label
Not whether a human appears somewhere in the diagram. Instead, for every consequential route:
Who reviews what, when does the action first take effect, what authority does the reviewer have, which errors can the evidence reveal, and how will decisions be explained, contested, and corrected?
Identical review of every route is neither necessary nor always desirable. What should be resisted is the blind branch: consequential action combined with no meaningful pre-effect review, no evidence adequate to the relevant error, and no practicable contestation.
If a workflow cannot answer that question route by route, “human in the loop” is a description of the org chart, not of a control.
The full article
This is a plain-language summary of a preprint. The full article develops the definitions formally, treats performance measurement under non-uniform verification in detail, works through the identification algebra that reconstructs a confusion matrix from branch volumes and representative verification, and sets out a six-part branch-first governance framework.
Human Presence Does Not Propagate: Asymmetric Human Oversight and Verification in AI Decision Systems George Evgrafov, QA4Tech. Preprint, 2026. https://doi.org/10.5281/zenodo.22658997
Open access, CC BY-NC-ND 4.0. Not peer reviewed.