Singapore put scam detection on the phone. The warning needs an error card

THE SIGNAL IN ONE SENTENCE

A scam warning is useful at the exact moment a person is about to do something expensive. Not six hours later in an email. Not next Tuesday in a police report. Right there, on the phone, before the bank transfer or the photo of an identity card leaves the device. Google DeepMind and Singapore's Government Technology Agency are trying to put that pause in the conversation itself. On October 14, Google described a system built with GovTech Singapore that can detect locally relevant scam patterns while messages remain on a person's device. It uses a lightweight first-stage classifier to screen incoming conversation. If that stage sees something suspicious, a fine-tuned Gemma model looks at a larger slice of the exchange and decides whether to show a warning. The warning advises the person to stop before sharing sensitive information or sending money. That is a sensible product shape. It also raises a very ordinary question that every safety system eventually meets: how often is it wrong? Singapore should publish an error card for the detector. The plain signal is that on-device AI can improve both timing and privacy. It can examine the conversational context that makes a scam convincing without first shipping that conversation to a government or company server. Google says neither it nor the Singapore Government can access a user's conversations through this deployment. Conversation access still depends on user consent and the terms of the application in which the detector appears. Those are meaningful boundaries. They are not performance evidence. The reviewed launch material does not publish a false-positive rate, a false-negative rate, supported phone models, language coverage, battery cost, latency target or a complete rollout schedule. Google calls the deployment carefully phased. That is more cautious than pretending the detector is already everywhere, but it means the public story is a plan and an architecture, not a measured national outcome. An error card would make that distinction visible. Start with the two kinds of mistake. A false positive is a legitimate conversation labeled suspicious. Perhaps an aunt asks for bank details to repay dinner. Perhaps a real employee sends an unusual payment request. Perhaps a charity volunteer uses urgent language because the deadline is, in fact, urgent. Too many false positives train people to ignore warnings. They can also chill ordinary communication, especially when the detector cannot explain which part of the exchange triggered concern. A false negative is the scam that passes through quietly. That failure may be more costly. A reassuring absence of warning can feel like proof of safety even when the software never promised that. The interface should say this plainly: no warning does not mean no scam. The error card should report both rates, not one blended accuracy score. It should break them down by scam type, language, dialect, code switching, message length, device class and warning threshold. It should show how results change when the scam relies on images, voice notes, links, calls or earlier messages the detector cannot see. One percentage point can hide several different products. A model that catches impersonation of a Singapore bank may miss a romance scam that unfolds over three months. A detector trained on English and Mandarin may behave differently in Malay, Tamil, Singlish or mixed-language chat. A phone with limited memory may receive a smaller context window than a newer device. Publish the slices. Then publish the window. Google says the second stage considers a larger snapshot of a multi-turn conversation. The size and selection of that snapshot matter. Does the model see ten messages, fifty or only the messages since the latest link? Can it inspect an image caption or a forwarded message? Does it know that the sender changed phone numbers? Can a user remove unrelated personal messages before asking for a check? Context is not a technical footnote. It is the evidence available to the warning. The detector also needs a consent receipt. On-device processing reduces exposure, but it still analyzes private conversation. A person should know which application is doing that, which message types are included, whether the feature is on by default, how to pause it and whether opting out changes any other service. Consent should be reversible without hunting through six settings pages. The receipt should distinguish inference from training. Google says training used anonymised datasets drawn from scam reports and intelligence feeds, and excluded private or personally identifiable citizen information. Fine-tuning happened in a GovTech-managed Google Cloud environment. That tells us something about model development. It does not mean a user's live messages become training data. The product should say whether they do, in one sentence, every time that policy changes. Next comes the model ledger. The detector will change. Scammers adapt quickly because adaptation is the job. A public card should name the model version, release date, supported applications, supported operating systems, supported languages, threshold, evaluation set and last update. It should summarize what changed after each update and whether performance improved for every important group. A safety model without version history is a weather forecast with no date. The ledger should include bypass testing too. Researchers should test deliberate misspellings, spacing tricks, coded language, screenshots, disappearing messages, slow-burn grooming, synthetic voice, deepfake video and requests that move a victim from one platform to another. They should test legitimate messages that resemble fraud, including emergency requests from family, payment reminders from small businesses and new account numbers from real suppliers. Singapore already knows that AI can strengthen the attacker. In an October 14 address at Singapore International Cyber Week, Minister for Digital Development and Information Josephine Teo described a deepfake-enabled scam in which one victim transferred nearly S$5 million. She said scammers are using AI to lower the cost of deception, impersonate people more convincingly and reach more victims. The same address said Singapore's Scam Analytics and Tactical Intervention System, or SATIS, now assesses more than 600,000 websites a day and disrupts around 20,000 malicious sites each month. A GovTech page last updated in May still describes more than 400,000 daily checks and about 15,000 monthly disruptions. That difference is not necessarily a contradiction. Systems scale and metrics change. It is a useful reminder that every number needs a date, definition and measurement method. The phone detector should not borrow SATIS results as proof of its own accuracy. Website scanning and conversational scam detection are different systems with different error surfaces. Singapore has another asset the warning can use: human help. ScamShield offers a 24-hour helpline at 1799. Its application and online tools let people check suspicious messages, links and numbers, report scams and filter calls and SMS. GovTech says the wider initiative has blocked or blacklisted more than 120,000 scam entities since 2022 and the app has more than 900,000 downloads. Put that help next to the warning. If the detector sees a risky exchange, the user should be able to tap for an explanation, send a report, preserve the relevant evidence and reach a trained person. The screen should not merely flash red and leave someone frightened, embarrassed and alone. It also needs an appeal path for legitimate messages. A bank, small business or family member whose conversation was repeatedly flagged should be able to report the mistake without teaching the system that all warnings are annoying. Reviewers should separate interface dismissal from ground-truth feedback. The person receiving a warning may not yet know whether it was correct. That distinction matters for learning. The detector should explain enough to support judgment without giving scammers a perfect recipe for evasion. It can say that the conversation contains an urgent request for money, a switch to an unfamiliar account and pressure to keep the transfer secret. It does not have to reveal every model feature or threshold. Useful explanation is a design problem, not an all-or-nothing disclosure fight. Older adults, new smartphone users and people with disabilities should be part of the rollout tests. Measure whether they understand the warning, whether screen readers present it correctly, whether the button labels are clear and whether fear causes them to close the app before reaching help. Test with people who have experienced scams. They know where shame and urgency defeat tidy interface assumptions. The system also has a physical cost. Running two models on a phone can consume memory, battery and processing time. Publish median and worst-case latency, energy use and storage by device tier. Say when the detector falls back to the lightweight stage only. Show whether older phones receive equivalent protection or a thinner version that deserves a different label. Privacy-preserving AI that works only on premium hardware can quietly become a safety divide. There is a responsible rollout hiding in all of this. Begin with volunteers in one application. Publish the model card and error card before expansion. Measure warnings, dismissals, confirmed scams, confirmed legitimate messages, help requests and abandoned conversations. Invite independent researchers to test the system. Create a process for reporting bypasses without publishing victim data. Review the results with consumer groups, banks, law enforcement and communities that use different languages. Then expand by evidence, not enthusiasm. On-device detection is not a magical referee living inside the phone. It is a fallible second opinion delivered at a useful moment. That can save people money and pain. The warning deserves trust proportional to the receipts behind it.