Britain and Canada renewed AI security cooperation. The shared tests need publishable results

THE SIGNAL IN ONE SENTENCE

International cooperation on AI security can sound like a very polished way to announce another meeting. Britain and Canada have at least chosen work that could become concrete. On October 13, the two governments renewed a partnership between the UK AI Security Institute and the Canadian AI Safety Institute. The agreement focuses on three areas: interoperable security standards, better information sharing and deeper technical cooperation. The last category includes regular exchanges about frontier capabilities and risks, possible staff exchanges and coordinated evaluations. Those are useful ingredients. They are not yet a meal. Neither government's announcement publishes a shared test suite, a model result, an incident threshold, a project calendar or a rule that says which findings will become public. The renewal is a plan for cooperation, not evidence that a particular model is safer or that the countries already agree on how to measure the hardest risks. That distinction matters because evaluation is where lofty AI policy either becomes operational or evaporates. Imagine two laboratories testing the same frontier model. One calls a cyber task a success when the model writes a plausible exploit. The other counts only an exploit that works in a controlled environment. One gives the model tools and several attempts. The other allows text only and one try. Both publish a score called cyber capability. The numbers may look comparable. They are not. An interoperable standard should make those differences visible. It should define the capability being measured, the model version, tool access, prompt policy, number of attempts, scoring rules, uncertainty and conditions under which a test must stop. If the method changes, the version should change too. This is the unglamorous work that lets one institution understand another institution's result. The renewed partnership says Britain and Canada will collaborate on standards that allow recognition and broad adoption of appropriate AI security practices. It also says the countries will identify priorities for future cooperation on common international standards. Good. But recognition should not become a rubber stamp. If one country accepts another country's evaluation, the public should know what was recognized. Was it the testing laboratory, the protocol, the result or a whole assurance process? How long does that recognition last? What happens when a model is updated, connected to new tools or deployed in a different environment? A model is not a toaster with one permanent safety certificate. Its behavior can change with system prompts, retrieval sources, tool permissions, fine-tuning, guardrails and software wrapped around it. A shared evaluation framework needs to describe the whole tested configuration, then say which changes trigger another review. Coordinated evaluations could reduce duplicated effort. The UK has invested heavily in model testing. Canada can bring a strong research network, bilingual and multilingual contexts, different public institutions and another set of experts asking awkward questions. A carefully designed joint exercise could expose weaknesses that either team would miss alone. It could also help smaller countries. Most governments do not have the budget, compute access or specialist staff to build a frontier-model testing program from scratch. A reusable protocol, reference implementation and non-sensitive example dataset would give them something sturdier than a press release and something cheaper than inventing an institute. This is why publishable results matter. Publishable does not mean dumping every dangerous prompt, exploit chain or private company record onto the open internet. The partnership explicitly plans to use appropriate Five Eyes channels for sensitive reporting. Some technical detail should remain protected when disclosure would make abuse easier, expose intelligence or reveal a live vulnerability before it is fixed. Secrecy can be justified. Vagueness cannot do all the work. The public layer should still say which risk category was tested, which protocol version was used, what kind of model and access were involved, how severe the finding was, whether mitigations changed the outcome and how much uncertainty remains. It should name the information withheld and explain the reason at a useful level. Think of it as a result card with a classified appendix. The result card could report that a model completed a defined class of cyber tasks above a stated threshold under controlled tool access. It could say that exact prompts and vulnerable targets are temporarily withheld. It could describe the mitigation and the date for another test. Researchers could scrutinize the measurement without receiving a ready-made attack recipe. That is a better public bargain than either extreme: publish everything recklessly or ask everyone to trust that unnamed experts saw unnamed evidence and felt reassured. The same rule should apply to societal resilience. The announcements say the countries will share lessons about building domestic resilience and strengthening their societies and economies. That phrase covers a lot of territory. It could include synthetic media, scams, attacks on public services, labour disruption or the concentration of critical technology in a few vendors. Each project needs a named harm, a population, an intervention and a measurable outcome. If a public agency tests a deepfake response exercise, report how quickly officials authenticated the media and reached affected communities. If it studies automated scam campaigns, report the languages and channels included. If it examines model use in critical infrastructure, publish the recovery assumptions and the boundary between a model recommendation and a human decision. Otherwise resilience becomes a compliment governments give themselves. Staff exchanges deserve a little paperwork too. Moving researchers between institutes can spread methods quickly. It can also create fuzzy lines around data access, conflicts of interest, publication rights and responsibility for incidents. Each exchange program should state what information visiting staff may access, how findings can be published, who owns tools developed during the placement and what happens when a researcher identifies a serious problem. The partnership should also invite independent scrutiny. Government institutes have access and authority that university teams often lack. Independent researchers have a different advantage: they can challenge the assumptions built into official methods. Both are needed. Before a shared protocol becomes an international standard, publish a draft and a reference implementation where security permits. Ask academics, civil society groups, model developers, affected workers and smaller governments to test it. Record substantive disagreements. Publish revisions with reasons. That process will be slower than declaring alignment. It will also produce alignment worth having. There is precedent for making at least part of this work public. Canadian government material says its institute has participated in multilingual evaluations led by Singapore and the UK, and jointly published model-testing results and a risk-assessment tool inventory at the Paris AI Action Summit in 2025. Canada also committed C$1 million in 2025 to the UK institute's Alignment Project. Those examples do not prove the renewed partnership will deliver transparent joint evaluations. They do show that collaboration can produce artifacts more useful than a memorandum. The next release should be a work plan. Name the first two or three evaluation areas. Publish owners, milestones and the protocol-development process. State which materials are intended for public release and which may remain protected. Define the threshold for notifying a developer, another government or the public about a serious result. Then publish a sample result card before the first high-stakes evaluation. It should include the model and configuration, test date, access level, evaluator, protocol version, task family, score distribution, uncertainty, mitigations, limitations and disclosure status. If a field cannot be public, mark it withheld rather than silently omitting it. The two institutes should also keep a revision history. AI evaluations age quickly. A benchmark can leak into training data. A model can learn to recognize the test. A capability that once required specialist prompting can become routine. The public record should show when a result expires, why a protocol changed and whether old scores remain comparable. None of this requires pretending that public evaluation alone secures a frontier system. Tests sample behavior. They do not prove the absence of danger. Models can behave differently outside the laboratory. A developer may patch one failure without fixing the underlying problem. Coordinated evaluation can improve the evidence available to governments, but it does not replace secure engineering, incident reporting, legal accountability or careful deployment. The renewed UK-Canada partnership is still worth doing. AI systems cross borders more easily than regulators do. Shared methods can help governments compare risks, avoid contradictory demands and recognize work performed by trusted partners. Sensitive reporting channels can help countries respond to threats that should not be broadcast in real time. The partnership becomes much more valuable when the non-sensitive layer is designed as public infrastructure. Publish the protocol. Version the tests. Describe the tested system. Separate measured fact from expert judgment. Give every result an expiry date. Explain what remains secret and why. Let outsiders reproduce the safe parts and argue with the assumptions. The plain signal is simple: two countries agreeing to test together is a promising start. The public benefit arrives when the agreement produces evidence that another laboratory, another government and an ordinary reader can actually inspect.