Canada and Germany backed an AI designed not to want anything

01

WHAT ACTUALLY CHANGED

Canada's September 16 announcement says the federal government plans to invest CAD 150 million through its Strategic Response Fund. The same release says Germany plans EUR 100 million, with that support subject to notification to the European Commission. These are separate commitments in different currencies, not one combined cash award. The announcement does not publish a payment schedule, contract, milestone table, clawback terms, or the amount already disbursed.

The Canadian money is intended to support talent and computing infrastructure for LawZero's Scientist AI program. The government says the project is expected to create 360 Canadian jobs and names Hypertec and 5C as computing partners. Those job, infrastructure, and sovereignty outcomes are projections attached to the investment. They require later evidence about hiring, retention, suppliers, capacity, access, energy, intellectual property, and who can keep using the work if a partner changes course.

Scientist AI is meant to behave as a disinterested predictor. LawZero's July paper describes a system trained to approximate a posterior over contextualized natural-language statements while avoiding a feedback loop in which the consequences of its deployed answers become the reward that shapes later answers. In plainer English, the proposed machine should estimate what appears true without secretly preferring one real-world outcome over another.

The architecture does not erase every form of agency. LawZero says any creative search or action should live in explicit, auditable scaffolding and be gated by a neutral estimator. That division is useful because it creates parts that can be tested separately. It also creates an integration problem: goals, permissions, tools, memory, and feedback can return through the surrounding software even when the predictor itself has no intended preference.

LawZero's published work offers semi-formal safety and accuracy arguments under assumptions about the data, contextualization, training process, estimator, and deployment. It is research evidence about a proposed design, not empirical proof that a completed Scientist AI is safe. Canada says the program will begin with tools for assessing existing AI and supporting science while moving toward safe-by-design frontier models. That sequence matters because the first useful result may be an evaluator rather than a new frontier model.

02

WHY THIS MATTERS

Most AI safety work starts with an agent that has already learned to pursue outcomes and then adds rules, evaluations, monitoring, and refusals. Scientist AI starts one floor lower. It asks whether a capable system can be trained to predict without acquiring a stake in the prediction. If the approach works, it could become a second opinion for scientific claims, risky plans, or other models. If it does not, the failure should teach researchers exactly which assumptions broke.

The distinction between intelligence and agency is practical. A calculator can answer without trying to make you keep using it. A navigation system becomes more consequential when it books, pays, contacts people, and adapts its strategy. Separating prediction from action can make permissions and failure points easier to inspect. It does not make the surrounding product harmless. Someone still chooses the objective, supplies the data, interprets the probability, and decides what the machinery may do.

A formal argument can expose assumptions more clearly than a promotional demonstration. That is a strength, not a magic trick. The public should be able to see which assumptions are mathematically required, which can be tested, which depend on inaccessible training data, which fail under distribution shift, and how much harm is possible when one is wrong. A theorem about an idealized predictor is not a warranty for a model, an application, or an institution.

Sovereign AI is often reduced to the location of servers. Real sovereignty includes control over data, models, skilled staff, energy, suppliers, evaluation, security, licensing, continuity, and the ability to move or stop a system. Canada and Germany can buy domestic compute and jobs while still becoming dependent on a narrow team, imported accelerators, private methods, or a partner that owns the crucial artifacts. Public investment should purchase durable public capability, not only local rack space.

This is also an industrial-policy experiment. Two governments are supporting an alternative technical path before a market has proved it. That can be a legitimate use of public research money when spillovers are large and private incentives favor faster products. The bargain needs measurable stages: open research, reproducible evaluations, workforce development, compute access, security, independent review, and clear consequences when a milestone is missed.

03

WHERE IT COULD HELP

• Publish one funding ledger that separates Canadian dollars from euros and records authorization, conditions, payments, recipients, procurement, compute delivered, jobs created, intellectual-property rights, missed milestones, clawbacks, and final public benefit
• Turn every safety claim into a testable assumption by naming the training objective, data transformation, estimator, feedback path, distribution limits, attack model, metric, threshold, independent reviewer, and consequence of failure
• Evaluate the predictor, neutral estimator, creative scaffolding, tool permissions, memory, and complete integrated system separately so a safe-looking component cannot lend its reputation to an unsafe product
• Give qualified outside researchers reproducible access to the relevant artifacts and the right to publish important positive and negative results, while documenting denied requests, conflicts, security limits, developer responses, and unresolved disagreements
• Define sovereignty through portability, governance, skills, supplier diversity, public-interest licensing, incident authority, energy and water disclosure, continuity plans, and the ability to stop or move the system without losing the work