Japan put a local-language model inside a doctor's evidence search. Accuracy is not clinical judgment

THE SIGNAL IN ONE SENTENCE

A citation can be real and still be the wrong answer. That is the useful tension inside a Japanese medical search tool called Evidence Finder. Sakana AI placed a case study on its public blog on October 9 describing how Aillis integrated the Japanese-focused Sakana Namazu model into Evidence Finder. The integration itself is older than the post. Aillis listed the adoption in a September 14 release. Evidence Finder first launched in May. That date distinction matters. This is not a brand-new clinical deployment arriving overnight. It is a newly detailed look at a product already being offered to physicians. The workflow is sensible on paper. A doctor asks a general medical question. Evidence Finder searches sources including PubMed and selects relevant literature. Namazu compares and combines information from those sources to produce an answer with citations. A Verify function checks whether the cited literature actually exists. This is exactly the kind of unglamorous assistance that could help. Medical evidence is scattered across studies, reviews and guidelines. The search terms are specialized. The paper that sounds decisive in an abstract may become much less decisive after somebody reads the methods. A tool that helps a physician find the right shelf faster is not pretending to invent medicine. It is helping with the library. But the library has traps. A real citation can study the wrong population. It can use a weak design, an outdated comparator or a surrogate outcome that does not translate into better health. It can conflict with a newer guideline. It can be retracted, corrected or too small to justify confidence. It can answer a nearby question while sounding as if it answered the actual one. Verification that a paper exists catches fabricated references. Good. It does not verify that the paper supports the sentence beside it. The plain signal is this: citation existence is a floor, not a clinical verdict. Sakana and Aillis also highlight exam results. Their company-run evaluation scored Namazu at 96.8 percent on Japan's 119th national medical licensing examination and 96.4 percent on the 120th. Image questions were excluded. Aillis says the latter was the best publicly available score it found among a specified group of domestic foundation models developed through the GENIAC program as of September 1. Those results are worth noticing. They also answer a narrower question than the product headline might suggest. A licensing exam asks whether a model can select answers from a defined test. Clinical work asks whether a professional can understand an incomplete story, examine a person, notice what is missing, reconcile conflicting evidence, communicate uncertainty and remain responsible when the stakes are not multiple choice. A high exam score is evidence of knowledge and reasoning on that exam. It is not evidence that patients get better or that dangerous errors become rarer. Sakana says this directly. Its own announcement notes that exam performance captures only one aspect of medical knowledge and reasoning and does not directly establish usefulness in clinical settings. That sentence should travel with the score. Evidence Finder's terms draw another boundary. The service is for licensed physicians and provides reference information by searching, summarizing and organizing general medical knowledge. It is not intended to diagnose, treat or prevent a condition in a particular patient, and the company says it is not a medical device. The terms also say the service does not guarantee completeness, accuracy, usefulness, recency or suitability, or that a displayed source supports a particular conclusion. In other words, the product's legal description is much more careful than the loose phrase "AI doctor." That is a feature, not an embarrassment. The useful model here is not machine replaces physician. It is machine shortens part of the evidence hunt, physician checks the evidence, and the product keeps enough provenance for somebody to reconstruct what happened. Provenance begins with the search. The interface should show which databases were searched, the exact query or query expansion, the search date, filters, language limits and the point at which the search stopped. Without that record, a polished answer can hide a narrow retrieval set. Then comes claim-to-source mapping. Each important sentence should point to the exact paper, guideline section or table supporting it. Opening the source should be one click, not a scavenger hunt. If several sources disagree, the answer should show the disagreement instead of averaging it into beige certainty. Study type belongs next to the citation. A randomized trial, observational study, case report, systematic review and professional guideline are not interchangeable tiles. Neither are a study of adults in one country and a recommendation for children somewhere else. The system should help the physician see population, setting, intervention, comparator, outcome, study size, publication date and major limitations. Recency is not just the year printed on the paper. Has a guideline changed? Was the article corrected? Has a later trial contradicted the result? Is the journal notice linked? A retrieval tool should check those conditions again when the answer is generated, not rely on an old embedding and a confident paragraph. Japanese context matters here. A local-language model may better understand Japanese phrasing, clinical shorthand and the way a doctor frames a question. It may be more useful with domestic guidelines and administrative language. Sakana says Namazu is adapted from the open Kimi K2.6 model to improve Japanese and Japanese business context while preserving general capability. That is a plausible advantage. It is not a blanket validation across specialties. The next evaluation should look less like one giant exam score and more like a cabinet of difficult drawers. Cardiology questions. Rare diseases. Drug interactions. Pregnancy. Pediatrics. Geriatrics. Japanese guidelines that differ from international recommendations. Questions with no good answer. Questions where the safest answer is to ask for more information. Questions where the literature is divided. For every drawer, measure more than whether the final prose looks right. Did the search retrieve the most relevant evidence? Did it miss a decisive source? Did the summary preserve uncertainty? Did every claim match its citation? Did the model confuse association with causation? Did it notice a retraction? Did physicians catch the error? How long did checking take? Did the tool reduce work, or merely move work from searching to debugging? The failure categories should be published, not just a single average. There is also a human-factors problem. A citation makes an answer feel researched. That visual cue can create more trust than the underlying evidence deserves. If the interface celebrates verification with a bright badge, a busy user may read "paper exists" as "claim confirmed." The label should say exactly what was checked. "Citation record found" is honest. "Verified" is dangerously broad unless the service also tested relevance, support, quality, recency and applicability. Hospitals and clinics considering this kind of tool can run a practical trial without pretending it is a clinical trial. Start with low-consequence evidence questions. Predefine specialties and question types. Compare retrieval against an independent search by librarians or clinicians. Record omissions, unsupported claims, stale guidelines and time saved. Require the physician to open the critical source before relying on the answer. Review incidents and near misses by version. Do not score only satisfaction. A fast answer can feel excellent while being wrong. Procurement teams should ask who owns the search index, how often it refreshes, which sources require licenses, how corrections propagate, how prompts and results are retained, where processing occurs, what happens when the model changes and whether the organization can export an audit record. They should separate the terms of the general model API from the terms of the specific Evidence Finder deployment instead of assuming one privacy description covers both. Clinicians need a smaller ritual at the screen: First, read the question the system actually answered. Second, open the strongest source. Third, check population, intervention, comparator and outcome. Fourth, look for newer guidance, corrections and conflicting evidence. Fifth, decide whether the source applies to the patient and setting. Sixth, document the human reasoning, not merely the generated summary. That sounds slower than accepting the paragraph. It is faster than repairing a decision built on the wrong paper. The diagram is simple: A physician asks a general evidence question. Evidence Finder searches and selects literature. Namazu synthesizes the retrieved material. The service checks that citations exist. The physician opens the sources and judges quality, recency and applicability. Only then does clinical judgment begin. The last step is not ornamental. It is the product boundary. This does not make Evidence Finder useless. Quite the opposite. Clear boundaries make useful tools easier to trust for the work they actually do. A calculator is valuable because nobody claims it examined the patient. A search assistant can be valuable because it can reduce the distance between a question and the evidence without pretending that distance is the whole practice of medicine. The Japanese-language work is also important beyond one product. Much of the generative AI boom has treated English performance as the main scoreboard and local adaptation as a footnote. Healthcare exposes why that order is backwards. Language carries clinical nuance, local guidelines, consent practices, administrative rules and the ordinary ways professionals ask for help. Local language is not a cosmetic layer. It is part of whether the system retrieves the right thing. Still, local fluency can make a wrong answer more persuasive. The better the prose, the more important the evidence trail. So keep the two achievements separate. One achievement is retrieval and synthesis in Japanese, with visible citations and a check that those citations exist. That can make evidence work faster and more accessible. The other achievement would be demonstrated safety and usefulness in real clinical settings across specialties, institutions and patient groups. That requires prospective evaluation, independent scrutiny, incident reporting and outcomes that matter to people. Sakana's own caveat and Aillis's terms already leave room for that distinction. Buyers, clinicians and readers should keep it open. The paper exists. Now read it. That is where the machine's answer ends and professional judgment earns its name.