Australia published an AI blueprint for government. Test the stop control before procurement

THE SIGNAL IN ONE SENTENCE

A building plan is not a building. It is a way to discover that the emergency stairs end inside a locked cupboard before somebody smells smoke. Australia's Digital Transformation Agency published an AI Reference Architecture on October 30 for agencies designing, buying, operating and governing AI systems. The architecture is voluntary. It does not approve a product, certify a model or replace the rules an agency already has to follow. What it does is unusually practical. It asks government teams to see the whole system before they commit to one attractive piece of it. That means the public-facing service, the workflow around it, the model, the data, the infrastructure, the vendor connections, the people supervising it and the evidence needed to show what happened. Not just the chatbot floating cheerfully on a procurement slide. The architecture is organized around four constructs. The first is a five-layer functional stack. The experience layer covers the people, devices and other systems that interact with the AI. The application and orchestration layer coordinates workflows, rules, models, human decisions and outside services. The intelligence and analytics layer covers how models are developed, tested, deployed, monitored and retired. The data and metadata layer covers collection, storage, preparation, sharing and lineage. The platform and infrastructure layer covers compute, storage, networks and hosting. The second construct is communication. Government systems have to connect old and new technology without leaking sensitive information or quietly losing control of data. The third is trustworthiness and assurance. The DTA calls for continuous monitoring and records of data, models and decisions. Agencies should be able to explain what a system does, test whether it remains reliable when conditions change, reduce bias, preserve accessibility, protect privacy and security, and consider long-term operating cost, environmental impact and dependence on outside providers. The fourth is human agency, oversight and control. This is where the architecture stops being a tidy stack of boxes. The DTA says people must be able to monitor, pause, change or stop systems within suitable timeframes. It also says pre-approved safeguards can act automatically, including pausing an operation or reversing a change when risk appears. Higher-risk or irreversible actions may need more review or escalation, and people should retain responsibility for final work. That is the signal. Government AI should not enter service until the agency has tested its stop control. Not confirmed that a button exists. Tested what happens when it is pressed. Imagine an AI system helping a call centre sort requests for a public benefit. It reads the message, classifies the issue, looks up records, drafts an answer and recommends the next step. Where is the boundary of that system? Does pressing pause stop the model from drafting? Does it also stop an orchestration service from sending the draft? Does it stop a vendor from retaining prompts? Does it freeze the queue safely or lose the cases waiting inside it? Can staff switch to a manual process? Can the agency undo a change already written to a case file? Who has authority to press the control at 2 a.m.? If the answers are scattered across five contracts and three teams, the agency does not have a stop control. It has a scavenger hunt. The DTA's companion guide makes this problem concrete through four representative scenarios: turning on AI inside an existing vendor platform, building an in-house service, procuring an agentic call-centre system and adding machine learning to case management. The guide does not pretend there is one correct design. It separates the controls an agency can govern, the behaviour it can configure, and the components it can design directly. That distinction matters because a team can be accountable for an outcome without controlling every technical layer. A vendor platform may let an agency choose permissions and retention settings while keeping the model, infrastructure and update schedule behind the vendor's curtain. An in-house system may offer more design control while adding operational responsibility. An agentic call-centre product may take actions across several services, making contractual limits and evidence especially important. A case-management integration can inherit old data, old permissions and old assumptions that a new model cannot magically repair. The right procurement question is not, "Do you have human oversight?" Every polished response to that question is yes. Ask for the drill. Choose a realistic failure: a model update increases false fraud flags, a data feed goes stale, a service starts exposing information across cases, an agent takes an action beyond its authority, or a vendor connection disappears. Then run the sequence. Who sees the first signal? What measurement crosses the alert threshold? Who can pause the system? Which components actually stop? What information is preserved? What public service continues? How are affected people told? Who decides to restore service? How is the change reversed? What evidence reaches the incident review? Time every step. The phrase "within appropriate timeframes" only becomes useful after an agency defines the timeframe for its own service. Stopping a drafting assistant in ten minutes may be acceptable. Stopping an automated payment or identity action in ten minutes may be ten minutes too late. The drill should begin with an architecture map that names the owner of every connection. For each layer, record the component, provider, data, purpose, authority, monitoring, failure mode and recovery path. Mark whether the agency can design it, configure it or merely govern the contract around it. Then draw the public-service fallback. If the AI stops, can a person still submit an application, receive urgent help, correct a record or speak to someone with authority? A safe shutdown that also shuts the public out is not a complete plan. The fallback does not need to reproduce the system's full speed. It does need to protect the essential service and identify who is most likely to be harmed by delay. Next, test the evidence. The DTA calls for records of data, models and decisions. That should include the model and version, system instructions, connected tools, data sources, retrieval results, important intermediate actions, confidence or risk signals, human approvals, final output, downstream change and the policy or rule in force at the time. Do not collect everything forever. Public agencies also have privacy, security, records-management and data-minimisation duties. The evidence plan should state what is logged, why it is needed, who can see it, how long it remains and how a person can challenge the record. The point is not surveillance by default. The point is reconstructability. If an agency cannot explain which system changed a person's case and why, oversight arrives after the evidence has evaporated. Procurement also needs an exit test. The DTA explicitly asks agencies to consider dependence on external providers. That dependence can live in model APIs, proprietary embeddings, identity services, data formats, monitoring dashboards, evaluation tools and staff knowledge. Before signing, export a representative package. Can another team recover the prompts, configurations, evaluation sets, logs, data lineage, access rules, model inventory and open cases in usable formats? Can the agency keep operating during a migration? Can it delete its information from the old provider and verify the deletion? Which licences or technical interfaces prevent a replacement? "The contract allows export" is not the same as "we successfully imported the export somewhere else." Run the move while the relationship is still friendly. The architecture's five layers also provide a useful way to keep evaluation from collapsing into model accuracy. At the experience layer, test whether people understand when AI is involved, can reach a person and can correct a result. Include accessibility testing with the people who use the service, not just a checklist. At the application layer, test workflow rules, permissions, escalation and tool boundaries. A capable model inside a reckless workflow remains a reckless service. At the intelligence layer, test the actual task, population and conditions. Measure error types, not only averages. Re-run the evaluation after model, prompt, retrieval or policy changes. At the data layer, trace origin, authority, quality, consent, transformations and known gaps. Mark when a source is stale or disputed. A model cannot restore meaning that the data pipeline removed. At the infrastructure layer, test outages, latency, capacity, security controls, backups, regional requirements and cost. Include the resource use of monitoring, duplicate systems and fallback capacity rather than counting only the model call. None of this guarantees a good public service. Architecture can become theatre too. A team can fill boxes with impressive nouns, draw arrows nobody owns and attach a risk register that has not changed since procurement. Voluntary guidance can be ignored. A scenario written by the government is not independent evidence that a commercial product is safe. A stop mechanism can work in a demonstration and fail under load, during an outage or after a vendor update. That is why the test must leave a receipt. Publish a compact version for significant public systems. Name the purpose, system boundary, accountable official, providers, high-impact actions, human review, last stop-control test, fallback service, major limitations and next review date. Explain what the public can do when the system is wrong. Some security details should remain protected. That does not justify hiding the existence of the control, the responsible agency, the date it was tested or the result. The architecture says it is intended for a range of adoption paths, from vendor-managed services to bespoke systems, and across machine learning, generative AI, agentic AI and emerging frontier capabilities. It is supposed to be used after the problem has been defined and validated. It does not replace the business case, procurement record, return-on-investment work or detailed implementation plan. That caveat is healthy. Sometimes the simplest compliant architecture is no AI. If ordinary software, a form redesign, better staffing or a cleaner data exchange solves the public problem with less uncertainty, the architecture should make that visible too. The best thing about Australia's blueprint is not that it puts every AI component into a government diagram. It is that it makes the missing door easier to spot. Now agencies have to pull the handle before the contract closes.