A bias audit and an ethics audit attach to different parts of a decision system, and that difference determines what each can catch. Mokander and Axente supplied the ethics-audit half of the distinction in November 2021 (arXiv:2111.04380). A bias audit, the kind a buyer commissions where a jurisdiction or a contract requires one, certifies disparity metrics computed on a frozen model at a single moment. An ethics-based audit, per their framework, traces intervention points across every phase of the software lifecycle. The practical consequence: a passed bias audit tells you almost nothing about downstream harm, so buyers should be contracting for pipeline instrumentation rather than certificates.
Where does each audit regime attach?
The two regimes are separated less by their values than by their attachment point: the bias audit attaches to a model artifact, the ethics audit attaches to a process. Mokander and Axente define ethics-based auditing (EBA) as “a structured process by which ADMS are assessed for consistency with relevant principles or norms,” and their central architectural claim is about where that process connects to the system under review. For EBA to be feasible and effective, they argue, its procedures must link to intervention points spanning all levels of organisational governance and all phases of the software lifecycle.
The bias audit, by contrast, attaches at exactly one point: a pre-deployment or periodically refreshed evaluation of a model’s outputs against protected-class categories. Everything upstream of the frozen model (problem formulation, data collection, labeling decisions) and everything downstream of it (deployment context, feedback loops, human override behavior) sits outside the audit perimeter by construction.
This is not a criticism of disparity metrics. Measuring outcome gaps on a fixed artifact is a tractable, repeatable, litigable procedure, which is precisely why regulators and procurement teams like it. The problem is scope confusion: buyers treat the artifact-level certificate as evidence about the system-level pipeline, and the certificate was never designed to carry that weight.
What does a moment-in-time bias audit actually certify?
A bias audit certifies that, on the day the samples were drawn, the frozen model’s outputs showed whatever disparity profile the audit measured. Nothing more. The model version, the test population, and the decision threshold are all fixed at audit time, and the resulting report describes that configuration.
Three properties follow from the mechanics. First, the audit is reproducible: rerun it on the same model and samples and you get the same numbers, which makes it attractive as a compliance artifact. Second, it is blind to everything dynamic: the production model drifts, gets retrained, or is quietly swapped by the vendor, and the certificate keeps describing a model that may no longer exist. Third, it says nothing about the humans around the model. If operators routinely override the system’s recommendations for one demographic and accept them for another, the deployed decision function differs from the audited model, and no output-sample audit will ever see that.
The paper itself supplies the premise for this gap, though from the other direction. Mokander and Axente contend that governance mechanisms currently used to oversee human decision-making often fail when applied to automated decision-making systems. The mirror-image failure is just as real: mechanisms built to oversee a model artifact fail when the harm lives in the socio-technical wrapper around it.
Which lifecycle stages does an output-sample audit never reach?
An output-sample audit reaches one lifecycle stage (post-training, pre-deployment evaluation) and misses every stage where harms actually accumulate: problem formulation, data assembly, deployment boundaries, runtime feedback loops, and override flows. That is the paper’s core structural argument, and it converts directly into a checklist.
Walk the lifecycle the way an intervention-point audit does:
- Problem formulation. The choice of what to optimize, for whom, and on which population. A hiring tool that optimizes for “retention at 12 months” has already made a consequential decision before any data exists. No model audit reaches this stage; the model does not exist yet.
- Data collection and labeling. Sampling frames, proxy variables, label definitions. A bias audit inherits whatever the data pipeline produced and can only measure disparities conditional on it.
- Training and pre-deployment evaluation. The one stage a bias audit covers. Disparity metrics on a frozen model live here, and only here.
- The deployment boundary. The gap between the evaluation environment and the live one: different population mix, different base rates, different decision stakes. A model fair on the test distribution can produce systematically different outcomes the day it ships.
- Runtime feedback loops. The system’s outputs change the population it later scores. A fraud model that concentrates investigations on one neighborhood generates more fraud records from that neighborhood, which justifies more concentration. This dynamic is invisible to any snapshot.
- Human override flows. Where operators accept, adjust, or discard model outputs, and with what consistency across decision-subjects. Override behavior is part of the deployed decision function and part of no bias audit’s scope.
- Change management. Retraining, vendor updates, threshold tuning. Each silently invalidates the last certificate.
The paper’s term for the connection points across these stages is “intervention points,” and the auditor’s job, in this framing, is to spark ethical deliberation at each of them rather than to render a verdict at one.
Why does socio-technical scope matter more than model scope?
Because an ADMS is not a model; it is a model embedded in an organisation, and the harms live in the embedding. The authors frame automated decision-making systems as parts of larger socio-technical systems, which is why they insist intervention points must span all levels of organisational governance, not just the engineering layer.
A standard dictionary sense of algorithm turns on the absence of case-by-case human judgment. Merriam-Webster’s definition of algorithm includes “a computerized process that evaluates data to perform actions (such as selecting or ranking items) without case-by-case human oversight.” Remove the human from the loop and the traditional accountability hooks (the manager who can explain a decision, the appeal that reaches a person) go with it. That is precisely the gap an audit has to fill, and a disparity report on a frozen model fills none of it. It cannot explain a single decision to a single decision-subject, because it contains no decision records at all.
Governance levels matter for the same reason. A disparity finding that reaches a data science team produces a retraining ticket. The same finding reaching a board, a compliance function, and a procurement team produces different interventions at different points: revised deployment criteria, contract clauses, escalation paths. The paper argues EBA should “inform, formalise, assess, and interlink” these existing governance structures rather than replace them, which is a deliberately unglamorous theory of change. The audit does not fix the system. It makes the system’s existing decision-makers unable to ignore what the audit found.
What standard does an ethics audit actually enforce?
Documentation sufficiency. The paper states that auditors’ main responsibility should be to spark ethical deliberation at key intervention points throughout the software development process and to ensure sufficient documentation to respond to potential inquiries. That is the whole enforcement mechanism, and it is worth pausing on how unlike a certification it is.
The authors describe EBA as a “soft” yet “formal” governance mechanism. Soft, because it does not gatekeep: there is no pass/fail stamp, no threshold, no certificate. Formal, because it is a structured process with defined outputs. The function of the audit is to trigger deliberation and to leave behind records good enough that someone later (a regulator, a plaintiff, a journalist, a decision-subject) can ask why a decision was made and get an answer.
A second paper by the same lead author, Ethics-Based Auditing of Automated Decision-Making Systems: Nature, Scope, and Limitations in Science and Engineering Ethics (Sci Eng Ethics 27, 44, 2021), specifies what such a procedure has to satisfy. It proposes seven design criteria for EBA procedures and sorts the constraints on implementing them into conceptual, technical, social, economic, organisational, and institutional categories. The sorting matters to a buyer: conceptual and technical constraints are research problems, while economic, organisational, and institutional ones are problems of contract and governance. Those are the categories a purchasing organisation can actually do something about.
This is also where the paper’s two-audience claim bites. EBA, the authors write, can help organisations verify claims about their ADMS, and it can provide decision-subjects with justifications for the outputs those systems produce. A bias audit serves the first audience only: it generates a document for the buyer’s compliance file. It structurally cannot serve the second, because a disparity metric is not a justification for any individual decision. If your obligation includes explaining outcomes to the people the system decides about, the artifact audit is the wrong instrument regardless of how rigorously it is executed.
Which audit type matches which failure mode?
Match the audit to the stage where your feared failure develops. If the failure mode is measurable in model outputs on a fixed population, a bias audit reaches it. If the failure mode develops at a deployment boundary, in a feedback loop, or in override behavior, only an intervention-point audit with pipeline instrumentation gets anywhere near it.
| Lifecycle stage | Failure mode that develops there | What a frozen-model bias audit sees | What an intervention-point audit demands |
|---|---|---|---|
| Problem formulation | Wrong objective, wrong target population | Nothing; the model does not exist | Documented rationale, deliberation record |
| Data collection and labeling | Skewed sampling, proxy variables, contested labels | Nothing; inherits the data as given | Lineage logs, sampling and labeling documentation |
| Training and pre-deployment evaluation | Disparate error or selection rates on the test population | The full audit; this is its entire perimeter | The same metrics, plus documentation of choices made |
| Deployment boundary | Context shift between test and live populations | Nothing; certificate predates production | Deployment criteria, environment diffs, sign-off records |
| Runtime feedback loops | System outputs reshape the scored population | Nothing; snapshot cannot see dynamics | Feedback-loop instrumentation, periodic re-measurement |
| Human override flows | Inconsistent overrides across decision-subjects | Nothing; operators are outside the artifact | Override records with rationale, reviewable per case |
| Change management | Retraining or vendor update invalidates prior certificate | A stale certificate describing a superseded model | Re-audit triggers tied to model and threshold changes |
Read the third column as the honest scope of the compliance artifact most organisations actually buy. Read the fourth column as a procurement specification, because everything in it is a data deliverable you either contracted for or will never get.
What should you demand from vendors at procurement?
Contract for the instrumentation an intervention-point audit consumes, at purchase time, as a delivery obligation. The brief list: lifecycle logs spanning development and deployment, override records with rationale, feedback-loop data, and documentation sufficient to answer inquiries about individual decisions. If the vendor’s system emits none of this, no audit regime can help you later, because there will be nothing to audit.
The paper’s policy recommendations double as a buyer’s checklist for audit ecosystems. The authors recommend that policymakers provide standardised reporting formats, facilitate knowledge exchange, offer guidance on resolving normative tensions, and create an independent body to oversee EBA of ADMS. A procurement team can impose the private-sector version unilaterally: demand standardised report formats in the contract, demand that audit findings reach governance levels above the engineering team, and prefer auditors accountable to something other than the vendor’s renewal cycle.
The constraint taxonomy in that Science and Engineering Ethics paper gives the checklist a second anchor, independent of the intervention-point paper’s policy recommendations. Of its six categories, the ones a procurement contract moves directly are the economic and organisational ones. Writing audit deliverables into the contract at purchase time is how a buyer clears the constraints the vendor has no incentive to clear.
The cost asymmetry is the argument for doing this at purchase time rather than after an incident. Instrumenting a decision pipeline during build is a feature. Retrofitting auditability onto a pipeline that logs nothing means reconstructing decision provenance the system never recorded, which in practice means it cannot be done at all; you can only start logging now and wait. Buyers who treat auditability as a post-hoc purchase discover that what they actually bought was a system with no observable internals and a certificate describing last year’s model.
Where does this framework fall short?
The framework is a 2021 conceptual paper that argues from structure rather than measured outcomes, and the regulatory specifics most readers want are not anchored here. Those limits deserve to be stated plainly rather than hedged away.
First, provenance. The arXiv page hosts a preprint, and arXiv’s own description is explicit that submissions are moderated for topicality and scholarly value but are not peer-reviewed, presented as-is without warranty. This particular paper carries a peer-reviewed journal reference, Mokander and Axente in AI & Society (2021), which puts it on firmer ground than the median arXiv find.
Second, evidence. The paper argues conceptually: its abstract outlines conditions for feasible and effective EBA procedures and mentions no case studies. No empirical case where an intervention-point audit caught a harm a bias audit missed appears in the material reviewed here, which is the abstract and metadata rather than the full text. The claim that a passed bias audit fails to predict downstream harm is the paper’s premise, reasoned from the structure of socio-technical systems, not a measured result. It is a strong premise; it is not a benchmark.
Third, anchoring. No statutory text and no published bias-audit report was reviewed, so this article deliberately contains no thresholds, dates, or penalties from any regime. If you need those, source them from the statutes, not from summaries of summaries.
The verdict survives those caveats. Judge audits by where they attach, not by what they certify. A moment-in-time disparity report on a frozen model cannot speak to harms accumulating at deployment boundaries, feedback loops, or override flows, and per the paper an audit changes a system only when it links to intervention points across all lifecycle phases and governance levels. Buy the bias audit if the regulation requires it. But contract for the instrumentation first, because the certificate expires with the next model update and the logs are the only part of the audit that compounds.
Frequently Asked Questions
How does the 2025 arXiv policy change affect the credibility of the Mokander and Axente framework?
The framework remains credible because it was published in AI & Society in 2021, predating arXiv’s November 2025 ban on unvetted position papers. The journal reference provides peer-review validation that the arXiv preprint alone lacks, ensuring the conceptual model is not subject to the recent platform restrictions on non-peer-reviewed content.
What specific contract clauses should buyers include to satisfy the ‘documentation sufficiency’ standard?
Contracts must mandate the retention of override rationale logs and model version timestamps for every automated decision. These clauses transform auditability from a post-hoc aspiration into a testable delivery obligation, ensuring that if a regulator or plaintiff inquires about a specific outcome, the vendor can produce the exact decision record and the human justification for any deviation from the model’s output.
Why is retrofitting auditability onto an existing unlogged pipeline considered technically impossible?
Retrofitting fails because it requires reconstructing decision provenance that was never recorded. You cannot log a human override that occurred six months ago if the system did not capture the operator’s identity or rationale at the time. The only viable path is to start logging immediately, accepting that historical data is permanently lost and that the audit trail will only cover decisions made after the instrumentation is deployed.
How does the ‘soft’ nature of ethics-based auditing differ from the ‘formal’ nature of bias audits in procurement terms?
Bias audits produce a binary pass/fail certificate that satisfies compliance checkboxes, while ethics-based auditing produces a deliberation record that triggers governance action. Procurement teams should not expect a ‘stamp of approval’ from an ethics audit; instead, they should contract for the specific governance interventions, such as revised deployment criteria or escalation paths, that the audit’s findings are designed to trigger within the organization.
