Japan listed its AI-for-science bets. Publish the milestones behind the titles

THE SIGNAL IN ONE SENTENCE

A project title can be a useful promise. It is not yet a result. Japan's Science and Technology Agency has posted the 31 projects selected for ARiSE, its AI for Science research program. ARiSE stands for AI to Redesign Scientific Exploration. The list is a striking map of where Japanese researchers want AI to enter the scientific process. There are materials systems meant to start discovery from scratch, a platform for hard metals and ceramics, a genome language model, immune and cancer digital twins, autonomous research facilities, an AI-enabled synchrotron platform, a tsunami world model, an RNA structure model and an international framework for robot-executable science. The international and convergence group reaches further into social-science replication, active experimentation in systems biology, quantum materials, fMRI, fusion control, gastrointestinal endoscopy, cybersecurity simulation, space science and global water prediction. That is the exciting part. The useful part is smaller and less glamorous. Most entries on the new project page still list the project overview as TBA. JST's selection notice, dated September 29, says 31 projects were chosen from 203 proposals. Seventy applications went to the strategic-target track and 133 to the international and convergence track. Twelve strategic projects and 19 international and convergence projects were selected after document and interview reviews supported by 82 advisers. The same notice says research is expected to begin after contracts and other arrangements are completed. So this is a funding and selection event. It is not evidence that an autonomous laboratory has discovered a material, a tsunami model has improved an evacuation decision or an RNA model has predicted a structure that survived experiment. Japan has published the bets. The next public document should be the milestone register behind them. That register should begin with the problem each project is actually trying to solve. The title "Tsunami World Model Project" is evocative. The public milestone needs to say whether the first task is faster simulation, better inundation estimates, scenario generation, decision support or something else. It should identify the geographic scope, the baseline model, the warning-time constraint and the errors that matter most. The same discipline applies to a genome language model. Is the first useful target sequence representation, function prediction, experimental design or a shared data layer? Which organisms and datasets are in scope? What counts as a meaningful improvement over a specialized baseline? Which predictions will face wet-lab validation? One title cannot carry all of that. The register should give every project a baseline. AI-for-science programs are especially vulnerable to the shiny-demo problem. A system can produce a plausible molecule, generate a tidy hypothesis or operate a robot arm without improving the scientific task that justified it. The baseline keeps the comparison honest. For materials discovery, compare against existing computational screening, expert selection and laboratory throughput. For a digital twin, compare predictive accuracy and calibration against current clinical or physical models. For an autonomous facility, measure completed experiments, valid measurements, recovery from failed runs and the amount of human intervention. A model that writes a cleaner lab plan but causes more invalid experiments is not scientific acceleration. Neither is a system that moves work from a scientist's bench to an invisible queue of human corrections. Next, name the public artifact. The artifact will differ by field. It might be a dataset, protocol, benchmark, model card, software release, machine-readable experiment description, instrument interface, negative-result archive or independently tested candidate material. The milestone should say whether the artifact will be public, shared under controlled access or kept inside a consortium. Restrictions can be legitimate, especially for patient data, cybersecurity and dual-use work. Silence is not a restriction policy. Record why access is limited, who can review the material and what public evidence can substitute for the protected data. Then publish the evaluation method before the result arrives. This matters because a broad portfolio cannot share one scoreboard. An RNA structure model may need blind prediction against newly measured structures. A tsunami system needs spatial and temporal error tests across events outside its training set. A social-science replication system must distinguish finding a paper, reproducing its analysis and establishing that its claim generalizes. A robot-executable science framework needs more than a successful demonstration. It needs protocol fidelity, cross-laboratory portability, machine-readable provenance, failure recovery and a clear record of every human correction. The register should freeze the primary measure, the comparison set and the test boundary before a team sees its final score. Otherwise, a program can quietly swap a difficult scientific goal for an easier technical proxy. Now add the dependencies. AI-for-science systems do not float above the research world. They depend on instruments, samples, databases, computing time, technicians, data stewards and partner institutions. The ARiSE list makes those dependencies visible in its team structures. Projects connect universities with RIKEN, the National Institute for Materials Science, the National Research Institute for Earth Science and Disaster Resilience, the Japan Synchrotron Radiation Research Institute, companies and international partners. That can be a strength. It can also create a milestone that looks complete in the model repository while the experimental or institutional work is months behind. For each milestone, show the data required, the compute and facility allocation, the instrument interfaces, the partner contribution and the dependency that is currently on the critical path. If a beamline, biobank or external laboratory is the real bottleneck, readers should not be staring at another parameter-count announcement. Safety needs its own column. The portfolio includes clinical data, genome models, autonomous facilities, cybersecurity simulation and scientific systems that could generate or execute experiments. The risks are not interchangeable. Patient-data work needs consent, privacy, representation and clinical-governance boundaries. Autonomous labs need physical interlocks, chemical and biological constraints, emergency-stop tests and authority limits. Cybersecurity research needs controlled environments and disclosure rules. Models that propose experiments need a human approval boundary that does not disappear when the system gets faster. Publish the boundary, the responsible person and the test that proves the boundary still works. Do not turn safety into a paragraph that says the team will be careful. The register also needs negative results. Science advances when a hypothesis fails for a useful reason. Program reporting often rewards the opposite. Teams polish a success path, omit the dead branches and make a complicated investigation look inevitable. That is especially dangerous with AI, where a team can try many prompts, checkpoints, datasets and evaluation cuts before one result looks impressive. Preserve the attempts. For every milestone, record the planned method, material changes, failed tests, excluded data, human overrides and decision to continue, revise or stop. A stopped branch can be valuable if it prevents other laboratories from paying for the same mistake. The final column should answer a blunt question. What would count as scientific progress here? Not infrastructure installed. Not a model trained. Not a robot connected. Those can be necessary milestones, but they are enabling work. Scientific progress might be a replicated result, a validated material property, an experimentally confirmed structure, a forecast that improves a real decision or a protocol that another laboratory can execute successfully. Keep infrastructure and scientific evidence in separate rows. That distinction would help people inside Japan evaluate the program without reducing a diverse portfolio to a single return-on-investment number. It would also help researchers abroad see where collaboration, replication and shared standards are actually needed. ARiSE is explicitly designed as a flagship program that concentrates investment in strategic areas and supports international collaboration. A public milestone register would make that ambition easier to trust. It does not need to reveal every unpublished hypothesis or sensitive dataset. It needs enough structure to show the public what changed, what remains uncertain and which evidence moved a project from one stage to the next. JST has already done the first accountable thing. It published the selected teams and their project titles. The project page also shows what is missing by placing TBA beside most overviews. Do not fill those blanks with marketing copy. Fill them with baselines, artifacts, tests, dependencies, safety boundaries and dates. The title is the bet. The milestone is the receipt.