Britain found more than 1,000 cyber flaws with AI. The scanners need a public scorecard

THE SIGNAL IN ONE SENTENCE

Britain has found a productive use for frontier AI: asking it to look for holes before attackers do. That is good news. It is also the beginning of the audit, not the end. On October 12, the UK government said two pilots had identified and addressed more than 1,000 potential vulnerabilities across government systems. Ministers now want to extend the program, use AI to protect essential services and develop better ways to respond when autonomous agents cause serious incidents. One of those pilots is unusually well documented. The Government Cyber Coordination Centre ran weekly, in-person hackathons that scanned public government code repositories. Teams tried several architectures. One used a six-stage chain in which agents challenged earlier findings. Another started with conventional scanners and used models to investigate attack paths. A third turned a large audit into reusable, domain-specific skills. The government case study says participants produced 407 findings across nine government organizations during one month. Some findings were already understood and controlled. Others were new. The case study says all critical weaknesses were remediated, no exploitation was found and model tokens cost £13,000. It also describes a serious example: a legacy GitHub Actions workflow could have let an outside user trigger remote code execution by placing crafted material in a pull-request comment. That is the useful part. The work moved beyond a synthetic benchmark and into real code, found exploitable paths, sent critical issues through existing patch processes and kept people in charge of escalation. The plain signal is that a vulnerability count is not a security outcome. A public scorecard should follow every expansion of the program. Start with the denominator. More than 1,000 potential vulnerabilities sounds large, but the public does not know how many repositories, services, lines of code or departments were scanned across both pilots. Without coverage, a count cannot show whether the scanner found a dense problem or merely looked at a large estate. Then separate candidates from confirmed flaws. The detailed pilot did some of this work. Human specialists checked model output, re-verified exposure and handled false positives. The case study says 407 findings included critical weaknesses, and that some findings were already covered by compensating controls. That distinction matters. A scanner can create thousands of plausible alerts and still make security worse if overloaded teams stop reading them. The most dangerous automated system is not always the one that misses everything. Sometimes it is the one that confidently fills every queue. Publish the number of raw candidates, validated findings, rejected false positives, duplicate findings, known issues and newly discovered issues. Break confirmed findings down by severity and exploitability. State which findings reached production code and which existed only in an unused path. Now measure repair. The ministerial speech says a central vulnerability-monitoring service reduced the time to fix identified weaknesses from 50 days to eight. That is a meaningful claim, but the published material does not define the measurement window, sample, median, average or severity mix. The scorecard should report time to human triage, time to containment, time to patch, time to retest and time to verified closure. A patch is not closed because a ticket moved columns. The affected service needs to be rescanned, the fix needs to survive deployment and the vulnerability must not return in a copied workflow three weeks later. Recurrence belongs on the same page. If an AI system finds the same unsafe pattern in twenty repositories, the answer is not twenty heroic tickets. The answer is a safer shared template, a blocked configuration and a test that prevents the pattern from coming back. That turns scanning into institutional memory. Human review should be measurable too. The case study says escalation was manual and that a team member checked every line in one agent pipeline. That is reassuring. It also creates a capacity question. How many analyst hours did 407 findings require? How many candidates could not be reviewed? Which decisions required a second reviewer? What evidence could an operator inspect before approving a patch? As the program moves from public repositories into closed-source systems, the scanners will gain access to more sensitive code, secrets, architecture and operational data. Their permissions should be narrower than the systems they inspect. Use read-only access by default. Isolate credentials. Limit network reach. Record every tool call. Require explicit human approval before changing production code or infrastructure. Keep rollback paths. Test the scanner itself for prompt injection, data leakage and attempts by repository content to steer the agent. Code can be hostile input. A comment, filename, documentation block or test fixture may tell an agent to ignore its task, reveal data or execute a command. A security scanner that obeys instructions hidden inside the thing it scans is an attacker wearing a high-visibility vest. The public scorecard should therefore include scanner incidents and near misses. The government is already considering how serious AI incidents and near misses should be identified and reported under a stronger cyber-resilience regime. Apply that logic to its own defensive tools. Count unauthorized access attempts, unsafe tool calls, leaked secrets, broken sandboxes, corrupted patches and cases where a model's confident explanation misled a reviewer. Do not wait for a public-service outage to invent the form. Independent testing matters because the government is both operator and narrator here. Give outside red teams a safe way to test the scanning architecture. Publish evaluation protocols, representative test repositories and versioned results where doing so does not create fresh risk. Let departments compare AI-assisted methods with conventional scanners and expert review. The comparison should ask a practical question: what extra verified risk reduction did the AI buy? Report critical flaws found only by the AI-assisted pipeline, critical flaws found only by conventional methods, reviewer time per verified issue, cost per verified issue and the proportion of fixes that passed retesting. Include misses discovered later. The £13,000 token bill is useful, but tokens are not the full cost. Count analyst time, model engineering, secure infrastructure, remediation work, incident handling and vendor dependence. Then compare that total with the value of faster discovery and repair. This is not an argument against scaling the pilot. The documented work has several encouraging traits. Teams used models as bounded components rather than magic oracles. Traditional tools fed model pipelines. Humans controlled escalation. Critical issues entered real remediation processes. The government disclosed an example, cost, participating organizations and several lessons. That is more evidence than most AI security announcements provide. The next phase can make the evidence durable. Publish a monthly scorecard with coverage, candidates, validated findings, severity, false positives, repair times, recurrence, human effort, scanner incidents and cost. Explain changes in scope so one month's total is not compared carelessly with another. Keep sensitive exploit details private while publishing enough aggregate evidence to show whether public risk is actually falling. Add a repair ledger for every critical issue. It can use anonymous identifiers. Record discovery, validation, containment, patch, retest and closure. If the same weakness returns, reopen the record. Add an authority ledger for the scanners. State what they can read, what tools they can call, what data may leave government systems, who approves an escalation and who can stop a run. Finally, add a miss ledger. When an incident reveals a vulnerability the scanners failed to find, record why. Perhaps the repository was out of scope. Perhaps the model missed a cross-service path. Perhaps a human dismissed the alert. Perhaps the flaw lived in configuration rather than code. Security improves when misses become training material, not embarrassment. Britain's experiment may be one of the more practical public-sector AI deployments now underway. It points models at a tedious, consequential problem and keeps expert judgment in the loop. The scanner is not the story, though. The story is whether hospitals, utilities and public services become measurably harder to break. Count that.