groundy
ethics, policy & safety

Public Sector AI Procurement Must Shift from Model Certification to Task Authorization

Public sector AI governance breaks under general-purpose models. One-time procurement certification fails because model behavior is not fixed. Teams must shift to task-level.

14 min···5 sources ↓

A preprint filed on 28 July 2026 argues that public-sector AI governance was built for narrow, single-task systems and breaks when applied to general-purpose AI, using policing as its case study. The practical consequence for procurement teams: a model card signed off at purchase no longer discharges the duty. Because a GPAI model’s behaviour is not fixed at deployment, control has to move downstream into contract prohibitions, task-level authorization, runtime controls, and continuous monitoring.

Why does general-purpose AI break narrow-AI governance assumptions?

General-purpose AI breaks narrow-AI governance because the properties that make it attractive to public services are the same properties that invalidate the conditions under which AI safety has historically been pursued. In the Relins preprint (arXiv:2607.25648), GPAI is defined as AI built on large language models that can be directed by prompt alone to perform an effectively unbounded range of tasks. Generality, accessibility, and low deployment cost are the selling points. They are also the failure mechanism.

The logic is worth stating precisely, because it inverts the usual procurement instinct. A narrow tool, say a number-plate recognition system or a benefits-eligibility classifier, has a fixed input domain, a fixed output space, and a task you can name in a contract. Every governance instrument public bodies actually use assumes that shape. You can test the system before purchase, certify it against the task, and treat the certificate as durable because the system’s behaviour does not change when a different user walks up to it.

A GPAI model has none of those properties. The same deployed system can draft a witness statement summary in the morning and answer an officer’s free-text question about charging standards after lunch, and neither use was specified, priced, or evaluated at procurement. The vendor sold a capability surface; the contract describes a product. That gap between what was bought and what can be done with it is where the paper locates the governance failure.

The paper also makes a subtler point about cost. Narrow AI safety worked partly because evaluation was expensive but bounded: you paid once to validate one task. When deployment cost drops to an API key and a prompt, the marginal cost of a new use case drops to roughly zero, while the marginal cost of governing that use case does not. Governance priced for a world of expensive, bounded deployments gets swamped in a world of cheap, unbounded ones.

What happens to accuracy, bias, explainability, and accountability under GPAI?

The four safety concepts that existing governance frameworks foreground (accuracy, bias, explainability, and accountability) each lose their operational meaning when outputs are unbounded, according to the preprint. This is the mechanical core of the argument, and each concept fails in a different way.

Accuracy. For a narrow system, accuracy is a number you can compute: error rate on a defined task against a labelled test set. For a model whose outputs span an effectively unbounded task space, there is no test set. Accuracy cannot be quantified over unbounded outputs because you cannot enumerate the outputs. A procurement team asking “how accurate is this model?” is asking a question that had an answer for the old kind of system and does not for the new one. The honest replacement question is “how accurate is it on this specific task, under these specific prompts, with these specific inputs?”, and that question has to be re-asked for every authorized use.

Bias. Bias auditing for narrow systems works by disaggregating error rates across protected groups on a fixed output: false-positive rates by demographic, say. When outputs are free-text judgements rather than classifications, disaggregation has nothing to attach to. A generated paragraph recommending for or against bail, or summarizing a suspect’s history, can carry bias in framing, emphasis, and omission in ways that do not reduce to a per-group error rate. The measurement apparatus assumes a confusion matrix that no longer exists.

Explainability. The preprint’s phrasing here is sharp: under GPAI, explainability becomes the appearance of explanation. A language model will happily produce a fluent rationale for any output, but that rationale is itself a generated text, optimized for plausibility rather than faithfulness to whatever computation produced the answer. A governance process that requires “an explanation” and receives a confident paragraph has satisfied its paperwork while gaining no actual insight. In a policing context, where explanations feed into decisions with legal consequences, the gap between a plausible rationale and a faithful one is not academic.

Accountability. Accountability erodes, the paper argues, as outputs are optimized to persuade. When the system’s output is designed to read as authoritative, the human reviewing it is working against the grain of the artefact itself. The chain of responsibility that accountability frameworks assume (developer, deployer, operator, each with a legible contribution) blurs when the operative decision was shaped by a prompt nobody logged and a response engineered to be convincing.

Taken together, the claim is that these four concepts were made tractable by narrow, purpose-built AI. They were not discovered to be properties of AI in general; they were properties of a particular class of system. Governance frameworks inherited them as if they were portable. Under GPAI conditions, each one needs to be rebuilt at the level of the individual task rather than the model.

Why is one-time procurement certification the wrong control for GPAI?

One-time, model-level certification is the wrong control because it certifies a fixed thing, and a GPAI model is not a fixed thing at deployment. Certification assumes the behaviour evaluated before purchase is the behaviour that will occur in production. That assumption held, roughly, for narrow systems. It fails for a model whose task is assigned by whoever writes the next prompt.

The timing mismatch compounds the scope mismatch. Procurement certification happens once, at acquisition. GPAI risk accumulates continuously, at each new use. A force that bought a certified system in January and whose analysts discovered in March that the same model drafts decent intelligence summaries has drifted outside its certified envelope without any procurement event occurring. No new purchase order was raised, so no governance gate was triggered. The system changed what it does without changing what it is.

The companion preprint by Salahuddin (arXiv:2607.25368), published the same day, gives this failure mode a name worth adopting: speed asymmetry. AI capability and its adoption inside an organization move faster than the governance instruments meant to contain them. The paper treats speed asymmetry as a structural construct, not a resourcing complaint, and its typology testing finds that none of the major frameworks address it at the operational specificity public-sector application requires. Procurement cycles measured in months cannot certify a system whose effective function changes in an afternoon.

This is why the certification model does not merely weaken under GPAI; it inverts. For narrow AI, safety assurance was an intrinsic feature of building the tool: you could not ship the classifier without deciding its task, and the task decision was the safety decision. The Relins paper argues that under GPAI, safety assurance becomes an optional add-on. The tool ships with its capability surface intact, and whether any particular use gets evaluated depends entirely on whether the deploying organization built a separate apparatus to catch it. Most have not.

Where should public bodies put their controls instead?

For GPAI, the working controls sit at the task and use level rather than the model level: procurement specifications that distinguish narrow from general-purpose systems, contracts that encode prohibited downstream uses, use-case authorization in place of model-level sign-off, and runtime controls with continuous monitoring carrying the safety load. This follows directly from the diagnosis. If the model’s behaviour is not fixed at deployment, the governance object cannot be the model.

Procurement specifications. The first control point is the taxonomy itself. The preprint recommends a clear distinction between narrow and general-purpose AI in governance documentation. This sounds administrative and is not. A procurement spec that does not record which class of system is being bought cannot attach the right obligations to it. Once the spec names the system as GPAI, the spec can also enumerate the authorized tasks, which converts the unbounded capability surface into a bounded, auditable list. The purchase decision becomes “this model, for these tasks” rather than “this model.”

Contract prohibitions. The second control point is the contract. If only enumerated tasks are authorized, everything else should be written into the contract as a prohibited downstream use, with consequences. This shifts part of the enforcement burden to the vendor relationship, one of the few channels through which a public body can actually constrain a general-purpose system it did not build. A model card cannot prohibit anything; a contract can.

Use-case authorization. The third control point replaces one-time sign-off with per-task authorization. Each proposed use of a GPAI system gets its own evaluation: what task, what inputs, whose data, what decisions does the output feed, what is the error cost. This is more expensive than a single certification, and that cost is the point. It re-imposes the bounded evaluation that narrow AI got for free. The companion typology paper frames the underlying risk as a governance vacuum: uses accumulating faster than any body has authorized them. Per-task authorization is the direct countermeasure.

Runtime controls and monitoring. The fourth control point is continuous: logging prompts and outputs, technical restrictions on what the deployed instance can be asked to do, and monitoring that treats drift into unauthorized tasks as an operational incident rather than a procurement footnote. This is the control most alien to traditional public-sector IT, because it treats an AI deployment less like purchased software and more like a production service with an abuse surface. Under GPAI conditions, that is what it is.

The following table maps the shift across the decision axes a procurement or audit team actually works through.

Decision axisNarrow-AI governance (existing)GPAI governance (required)
Scope of what is governedThe model and its fixed taskEach authorized task and use case
Timing of controlOne-time certification at procurementContinuous, per-use authorization plus monitoring
Control locationModel card, pre-purchase evaluationContract prohibitions, runtime use-case controls
Accuracy evidenceTest-set error rate for the fixed taskPer-task evaluation; no model-wide number exists
Bias evidenceDisaggregated error rates by groupTask-level judgement audits on free-text outputs
Accountability basisFixed developer/deployer/operator chainPrompt-level logging and per-use decision records

Which existing frameworks cover AI governance failure modes?

None of the five major frameworks tested addresses AI-driven cyber governance failures at the operational specificity public-sector application requires, according to the typology analysis in arXiv:2607.25368. The paper proposes a seven-domain typology of ten AI-driven cyber governance failure causes in public-sector organizations, then tests five instruments against it: NIST CSF 2.0, ISO/IEC 27001, COBIT, NIST AI RMF, and ISO/IEC 42001 (arXiv:2607.25368).

Three gaps survive across all five. Shadow AI: staff using models outside any managed channel, invisible to the governance apparatus by definition. Speed asymmetry: capability and adoption outrunning the governance cycle. Governance vacuum: the accumulation of live, ungoverned use cases with no owner. A framework can be fully implemented, audited, and certified against and still leave all three open, because the frameworks were written against a threat model in which the organization knows what systems it runs.

The same paper models how failures compound: a three-pathway structure in which accountability failure, operational resilience failure, and compliance failure interact and reinforce each other. That interaction matters for how a public body reads a clean audit. Passing ISO/IEC 27001 says something real about information-security management. Under the typology’s model (arXiv:2607.25368), it says little about whether an ungoverned GPAI use case is quietly feeding citizen data into a third-party API, because that failure lives in the gap between the instrument’s categories.

Two cautions apply to this coverage finding. First, it is a typological analysis, a structured argument about what the frameworks’ texts address, not an empirical study of deployments that failed while certified. Second, the paper’s own confidence profile should be carried with it: the mapping of specific gaps is presented as the authors’ assessment, and the brief supporting this article rates that portion medium-confidence. The finding is best read as a hypothesis with a clear structure, one that any public body can test cheaply against its own framework implementations by asking where Shadow AI, speed asymmetry, and use-case ownership actually appear in its documentation.

What does the policing paper recommend?

The Relins paper makes four recommendations: a clear taxonomic distinction between narrow and general-purpose AI in governance documentation, a preference for technological parsimony, a pause on operational GPAI deployment in policing until adequate evidence exists, and a coordinated national safety infrastructure with authority to generate that evidence. Each deserves a plain reading.

The taxonomy recommendation is the cheapest and the most enabling, for the reasons covered above: nothing downstream works if the documentation cannot say which kind of system it governs. Technological parsimony is the more contentious one. It amounts to a default preference for the narrow tool when a narrow tool exists for the task, on the grounds that narrow tools are the ones governance can actually hold. That runs directly against procurement economics, where one general model is cheaper than five purpose-built systems, and against vendor sales motions, which sell the capability surface precisely because it is unbounded. Parsimony is the recommendation most likely to be adopted in documents and ignored in purchases.

The deployment pause is the strongest recommendation and the one most likely to be quoted out of context. It is scoped: operational GPAI deployment in policing, until adequate evidence exists. It is not an argument against GPAI research, evaluation, or use in low-stakes administrative work. The reasoning follows from the safety-concept collapse: if accuracy, bias, explainability, and accountability cannot currently be established for GPAI outputs, then deploying those outputs into decisions with coercive legal consequences means running the experiment on the public. The paper’s position is that the evidence should come first.

The national safety infrastructure recommendation addresses who generates that evidence. Individual forces or departments cannot each build the evaluation capacity to quantify GPAI behaviour per task; the costs that make per-use governance expensive at the organizational level argue for pooling it. The paper proposes a coordinated body with the authority, not merely the advisory role, to produce the missing evidence base.

For readers working through EU AI Act obligations, including the Annex III provisions covering law-enforcement AI: the research underlying this article did not verify the Act’s specific deadlines or conformity requirements against a primary source, so this article deliberately makes no claim about them. The compliance mapping is real work that needs doing, but it should be done against the regulation’s text, not against a preprint’s framing. Any statement here tying the paper’s recommendations to specific Annex III obligations would be unverified, and is therefore omitted rather than hedged.

How much weight should these findings carry?

Both anchoring papers are preprints, and that status bounds what a practitioner should do with them. Per arXiv’s own description, submitted material “is wholly the responsibility of the submitter and are presented ‘as is’ without any warranty or guarantee.” The Relins preprint has not completed peer review; the Salahuddin typology has been accepted at the ML4CS conference, but neither paper presents empirical deployment data. The Relins paper is a single-author normative argument built on one case study; the Salahuddin paper is a typological analysis whose framework-coverage findings are structured argument rather than observed failure.

arXiv:2607.25648 is marked as submitted for peer review, and its core mechanism, that unbounded outputs defeat bounded evaluation, is close to self-evident once stated. That does not make the argument wrong, but it does mean the recommendations should be treated as directional, not validated.

The strongest limitation is the single case study. Policing is a demanding case: high-stakes decisions, legal exposure, coercive outcomes. Governance that fails there might hold in lower-stakes public services, or fail differently. The paper asserts the misfit is likely to recur elsewhere but does not demonstrate it. A reader in benefits administration or healthcare procurement should import the analytical frame (does my governance assume a fixed task?) rather than the policing-specific conclusions.

The verdict, then. The diagnosis is strong and should change behaviour now: if your governance instruments certify models at procurement, and you are deploying general-purpose models, your governance is measuring a property the system does not have. The remedy direction is also sound: task-level authorization, contract-level prohibitions, runtime controls, and continuous monitoring follow logically from the diagnosis and cost little to begin against the taxonomy step. The specific recommendations, a policing deployment pause and a national safety infrastructure, are normative claims from one preprint and should be weighed as such. What does not survive scrutiny is the status quo: a one-time procurement check presented as assurance for a system whose behaviour was never fixed at purchase. That check discharges nothing.

Frequently Asked Questions

Does the Relins paper recommend pausing all AI use in policing?

No. The recommendation is scoped to operational general-purpose AI deployment. It does not cover narrow, purpose-built systems like number-plate recognition, nor does it block research, evaluation, or use in low-stakes administrative work where the risk profile is lower.

How does the Salahuddin typology define speed asymmetry?

It defines speed asymmetry as a structural construct where AI capability and organizational adoption outrun the governance cycles designed to contain them. The analysis finds that procurement cycles measured in months cannot certify a system whose effective function changes in an afternoon, leaving a gap that existing frameworks do not address.

What is the practical first step for a public body adopting this framework?

The first step is taxonomy: classify every AI system in the estate as narrow or general-purpose. This inventory is the precondition for contract prohibitions, per-task authorization, and monitoring. It is achievable with existing staff before any new framework is adopted or any new vendor contracts are signed.

Why does explainability fail under general-purpose AI conditions?

Explainability becomes the appearance of explanation. A language model will produce a fluent rationale for any output, but that rationale is itself generated text optimized for plausibility rather than faithfulness to the underlying computation. In policing, where explanations feed into decisions with legal consequences, the gap between a plausible rationale and a faithful one is critical.

What are the three gaps that survive across all five major governance frameworks?

The three gaps are Shadow AI, speed asymmetry, and governance vacuum. Shadow AI refers to staff using models outside managed channels. Speed asymmetry is the mismatch between adoption speed and governance cycles. Governance vacuum is the accumulation of live, ungoverned use cases with no owner. These gaps persist even in fully implemented and audited frameworks.

sources · 5 cited

  1. About arXiv - arXiv infoinfo.arxiv.orgprimaryaccessed 2026-07-31
  2. ArXiven.wikipedia.organalysisaccessed 2026-07-31
  3. NIST AI RMFnist.govvendoraccessed 2026-07-31