Paris promised a shared AI safety repository. Publish the evidence format

THE SIGNAL IN ONE SENTENCE

Paris has a new AI safety promise. Now it needs a filing system. On November 9, a group of governments, frontier AI companies and civil-society organizations signed the Paris Declaration on AI Safety and Security. Official pages published by the governments of the Netherlands and Canada describe a common agenda for evaluating advanced models, sharing non-sensitive results, reporting incidents and protecting people who raise alarms. The declaration is unusually concrete in a few useful places. It calls for a common framework for frontier-model evaluation and risk management. It proposes standardized publication of non-sensitive evaluation results in a shared public repository. It supports independent third-party evaluation, stronger information sharing, common incident classification, rapid alert systems, protected whistleblower channels and fail-safe shutdown mechanisms. It also names hard problems instead of pretending they have been solved. The declaration points to cyber and biological risks, models operating with greater autonomy, misalignment, loss of control and techniques that can remove or bypass safeguards. It says safety work should preserve the benefits of open-weight models rather than treating public access as a category error. That is the verified reporting. Here is the analysis: the repository could become the most useful part of the declaration, or the quietest. A shared repository is not automatically shared evidence. If every laboratory uploads a different kind of summary, uses a different model name, hides the test boundary and reports only the most flattering aggregate, readers receive a brochure shelf with a search box. The public needs a common evidence format before the shelf fills up. Start with the identity of the thing that was tested. An evaluation record should name the exact model version, checkpoint or release candidate. It should identify the weights, system prompt, tool permissions, inference settings, safety layer and deployment configuration that shaped the result. If the public product changes after the test, the repository should say whether the evaluation still applies. Model families are slippery. A name can survive while routing, safeguards, tools and hidden instructions change beneath it. A result attached only to the family name is a weather report without a city. Next, name who designed and ran the test. The declaration supports independent third-party evaluation. That word independent needs fields, not applause. Who paid the evaluator? Who selected the tasks? Could the developer veto publication? Did the evaluator receive the same access as the developer? Was it allowed to test the model with tools, long contexts, repeated attempts or fine-tuning? Were conflicts of interest disclosed? Independence is not a personality trait. It is an arrangement that can be inspected. Then publish the evaluation boundary. Every result should state which capability or hazard was tested, which populations and languages were included, which threat model applied and which conditions were out of scope. It should preserve the test-set version, provenance, contamination checks, prompt budget, attempt limit, grader and scoring rule. This matters because one clean percentage can hide a small, friendly test. A model that refuses a dangerous request on the first try may comply after a harmless preamble, a long conversation, a tool call or twenty attempts. A cybersecurity result without the available network, credentials and targets is just a number wearing a lanyard. Thresholds belong in the record too. Before the result arrives, what outcome would trigger more safeguards, a delayed release, a restricted deployment or a stop? If the threshold appears only after the score is known, it is not a gate. It is interior decorating. The repository should distinguish raw evidence from conclusions. Publish aggregate results, subgroup results, uncertainty and failure examples where doing so is safe. Record how many runs were performed, how much human help was provided and whether another team reproduced the result. Some details cannot be public immediately. A live cyber exploit, dangerous biological procedure, personal information or active security vulnerability may require redaction. Redaction should be a documented decision, not a magic curtain. The public record can still identify the withheld category, reason, decision maker, review date and conditions for later release. A trusted regulator or independent evaluator may receive the full material under controlled access while the public receives a safe account. The repository should preserve the existence of the evidence even when the evidence itself cannot be posted. Version the record when the model changes. If a developer adds a safeguard, removes a capability, changes the system prompt or expands tool access, the entry should link the old and new evaluation. The repository should show which failures were remediated, which remained open and when the next test is due. This is especially important for open weights. The declaration says the framework should preserve the benefits of open-weight models while addressing techniques that can remove safeguards. That is the right tension. A public model is not one fixed service. People can run it with different prompts, quantizations, adapters, tools and safety layers. The evidence packet should separate base-model capability from a particular deployment wrapper and test realistic modifications without assuming that every fork is malicious. Do not turn the repository into a license for broad restrictions that the evidence does not support. Publish the test conditions, the observed failure and the specific risk boundary. Let researchers challenge the result. Let open-source communities submit reproductions and mitigation evidence. A shared system should make disagreement visible, not make one institution's conclusion permanent. Incident reporting needs a connected record. The declaration calls for common incident classification and rapid alert systems. When a model causes or nearly causes a serious event, the alert should link to the evaluated model and the safeguards that were supposed to prevent it. Record when the event began, when it was detected, when the developer understood its significance, when affected parties were told and when a public account appeared. Name the systems touched, the authorized scope, the observed consequences, the uncertainty and the immediate containment step. Preserve near misses too. A narrowly avoided failure can reveal a broken assumption before someone pays the full price. The rapid alert channel should not wait for a polished investigation. Use a tiered process. Send a protected early notice to the relevant authorities and affected operators. Share technical indicators with trusted responders. Publish a safe preliminary account. Update the repository as facts change. Mark corrections instead of quietly replacing the old text. Whistleblower protection belongs beside this workflow, not in a separate values paragraph. A reporting channel should accept concerns from employees, contractors, evaluators and affected users. It should explain confidentiality limits, anti-retaliation protections, response deadlines, escalation routes and how a reporter can preserve evidence without exposing sensitive material to the wrong audience. The repository should record that a protected report exists without naming the reporter or revealing the substance prematurely. It should also record whether the concern led to an evaluation, mitigation, incident review or no action, with a reason that can survive public scrutiny. Fail-safe shutdowns need evidence as well. The declaration calls for shutdown mechanisms where appropriate. That phrase covers a lot of territory. A laboratory training run, a research inference cluster, a public API and an autonomous tool-using agent do not share one stop button. Publish the shutdown boundary. What can be stopped? Who can authorize it? How long does the intervention take? What happens to queued work, cached outputs, external tools and already issued credentials? Does the system fail closed when the control service is unavailable? When was the mechanism last tested under load, during a network failure and against an unauthorized restart? A red lever in a diagram is not a fail-safe. A dated drill with measured results is closer. The shared repository can connect these pieces into one public chain: model identity, evaluation scope, threshold, result, redaction, decision, deployment, incident, remediation, retest. That chain would help more than researchers and regulators. Buyers could compare models without pretending vendor scorecards are interchangeable. Journalists could see whether a safety claim refers to the current release. Workers could check whether a tool deployed in their workplace was evaluated for the tasks it actually performs. Open-source maintainers could distinguish reproducible findings from vague warnings. Governments could coordinate without forcing every country to rebuild the same filing cabinet. The declaration itself is not a treaty, and the official pages do not provide a binding timetable, repository operator, mandatory schema or enforcement process. The Canadian statement says the declaration remains open to endorsements and will be updated. The signatory lists on the Canadian and Dutch pages are not identical, which is a small but useful preview of why versioned public records matter. So treat the declaration as a work order. Within ninety days, publish a draft schema. Run it against several existing model evaluations. Invite comments from independent evaluators, open-source developers, workers, affected communities, security responders and researchers outside the main laboratories. Show which fields are mandatory, which may be redacted and which authority resolves disputes. Then test whether a stranger can answer basic questions from the repository. Which exact model was evaluated? Who ran the test? What could it do? What failed? What threshold applied? What changed before release? Which facts are withheld, and why? Has an incident contradicted the original safety case? When will the model be tested again? If the repository cannot answer those questions, it is not yet public safety infrastructure. Paris produced the promise. The useful sequel is a boring, versioned, inspectable form. That is how shared concern becomes shared evidence.