Physical agents improved after failure. Keep the workspace in the receipt

THE SIGNAL IN ONE SENTENCE

A robot drops the block. That failure can disappear into a final success rate, or it can become part of the system that makes the next attempt better. PhysRSI-eval, an October 19 technical report from the Galbot Team, is interested in the second possibility. The authors evaluate six vision-language systems on 18 dual-arm manipulation tasks and let each system revise a persistent workspace between attempts. The model weights stay fixed. The files around the model change. That distinction is the heart of the paper and the practical lesson worth carrying away. The plain signal is this: when a physical agent improves after failure, preserve the workspace that changed, not just the score that rose. The workspace can contain memory, reusable skills, tools and policy code. After each physical attempt, the system receives observations and a task score. Robot actions are then disabled during a revision phase. The agent can inspect what happened, edit its files and prepare the next attempt. The evaluation allows up to 17 refinement rounds under a fixed interaction budget. This is not model retraining. The six systems use fixed model weights throughout a task. It is closer to an engineer keeping a lab notebook, editing a control script and reusing a calibrated tool after watching a machine fail. The benchmark uses 18 tasks from RoboDojo, a dual-arm manipulation environment. The task set spans six difficulty strata, with three tasks in each stratum, across five official categories. The main evaluation uses one layout and one seed for each task. Step limits range from 300 to 1,300 simulator steps at 25 hertz. The systems include GPT-6 Astra, GPT-6.1 Sol, GPT-6 Sol, GPT-5.6 Sol, Claude Opus 5.5 and Claude Sonnet 5.5. The paper sensibly describes each result as a system result, because the runtime configuration sits beside the model. That is important. A model name alone does not tell you which tools, memory rules, prompts or execution controls produced a result. PhysRSI-eval reports five axes instead of collapsing everything into one leaderboard number. The first axis measures performance on the first attempt. The second measures how early gains arrive across rounds. The third measures the best attained score. The fourth measures how much of the allowed physical interaction budget remains when success arrives. The fifth measures how often an initial failure becomes a later success. Those axes answer different operational questions. A system can eventually finish many tasks but burn through attempts while doing it. Another can begin well but fail to turn its remaining mistakes into improvements. A third can reach a high partial score without completing the task. One final average would hide all three patterns. Across the 18 tasks, the authors report that task-mean scores rose from a range of 0.01 to 0.37 on the first attempt to 0.10 to 0.84 at the end of the allowed process. GPT-6 Astra solved 15 of 18 tasks, Claude Opus 5.5 solved 13, GPT-6.1 Sol solved 12, Claude Sonnet 5.5 solved 11, GPT-6 Sol solved seven and GPT-5.6 Sol solved one. Astra led the five-axis profile in the authors' evaluation. Opus illustrates why the profile matters. Its reported success score was 0.744, slightly above GPT-6.1 Sol at 0.703. But its efficiency score was 0.609 versus 0.822, and its budget score was 0.579 versus 0.808. Similar eventual performance can arrive through very different amounts of physical trial. The paper also reports exploratory associations between benchmark performance and other capabilities. Spatial reasoning had the strongest observed association with first-attempt performance and efficiency, while perception had the strongest association with attained score and improvement from failure. Coding showed weaker associations in this small comparison. That is correlation across six systems, not a causal explanation. The report calls the analysis observational, and six points are nowhere near enough to turn a tidy coefficient into a law of nature. The stronger evidence for the value of history comes from a smaller controlled ablation. The researchers selected six tasks, ran two chains per task and changed the layout on every refinement round from K1 through K6. With the full loop, Astra succeeded on 15 of 36 late-round attempts, or 41.7 percent. When prior history was removed, it succeeded on three of 36, or 8.3 percent. Removing grader feedback produced the largest reported drop among the withheld information types, to five successes out of 36. Removing earlier images produced 12 successes. A setup that kept the workspace and grader feedback while withholding the rest of the raw record produced 17 successes, slightly above the full-loop count in this limited sample. That last result is a useful nuisance. More stored context is not automatically better context. A crowded history can distract a system, while a compact workspace plus specific feedback may preserve the part that matters. The benchmark does not establish one universal memory recipe. The workspace also carried some value into unfamiliar layouts. After six refinement rounds, the researchers froze Astra's workspaces. They then tested five unseen layouts without allowing further revision or feedback. Workspace files alone succeeded on 25 of 60 trials, or 41.7 percent. Adding raw records produced 23 successes, or 38.3 percent. That is evidence of retained utility, not proof of broad physical understanding. The held-out layouts remained inside the benchmark task family. The source and held-out comparisons also used different denominators, so they should not be treated as paired reliability estimates. Cross-task transfer is more preliminary still. The report describes three first-run cases in which a workspace from one task moved first success earlier on another task, from round K9 to K1, K14 to K7 and K15 to K8. Three cases are interesting enough to test again, not enough to promise general transfer. The tool traces show that systems may improve through different workspace strategies. Claude Opus 5.5 and Sonnet 5.5 used self-written tools for more than 96 percent of later-round actions. GPT-6 Sol used tools for about 67 percent and direct action generation for the rest. Astra and GPT-6.1 Sol generated about 99 percent of their actions directly while consulting memory, skills and policy notes. So a durable workspace is not one specific folder structure. It is an accountable layer where a system can retain the parts of an attempted solution that its runtime knows how to use. That layer needs a receipt. For every physical attempt, record the task and success condition; the initial layout, seed and object state; the model and runtime configuration; the allowed tools; the observation stream; the complete action trace; the native score; the reason for failure; the grader feedback; the exact workspace snapshot before and after revision; files created or changed; tool and policy versions; step budget used; revision time; token and compute costs; the stop reason; and any later task or layout that receives the frozen workspace. The benchmark's physical-budget axis counts canonical interaction steps. It does not include wall-clock time, model tokens, money, energy, machine wear or the safety cost of repeated contact. A deployment receipt should. Version each round as an immutable snapshot. Give files stable identifiers. Record lineage when a tool, memory note or policy fragment is copied into a later workspace. Keep the failed snapshot even after a successful revision. Otherwise an investigator sees only the cleaned-up answer and cannot tell which observation corrected the system. Separate three questions during evaluation. Did the next attempt improve on the same layout? Did the frozen workspace retain value on an unseen layout without fresh revision? Did anything transfer to a different task? Those are progressively stronger claims. Do not let the first quietly stand in for the third. For a warehouse gripper, the receipt might show that a failed grasp exposed an occluded edge, prompted a camera-angle rule and improved a later attempt. For a laboratory robot, it might show that a collision caused a slower approach policy and a new fixture check. For a field robot, it might show that a wheel-slip note helped on another patch of the same terrain but failed after rain. Those are Plain Signal examples, not results from the paper. They illustrate why transfer needs an explicit boundary. The safest promotion path is boring on purpose. Freeze the revised workspace. Test it without further editing on several unseen layouts. Repeat successful runs to estimate reliability, because PhysRSI-eval stops a chain at first success. Add regression tasks so a new tool cannot quietly break an old capability. Put hard force, speed and workspace limits outside the agent. Require a person to approve consequential policy changes and any move from simulation into a physical environment. PhysRSI-eval has real limits. It is an author-run technical report with one improvement chain per task. Repeated first attempts do not estimate the variability of full improvement chains. The benchmark stops after first success, so it does not ask whether the system can repeat that success. Conditional task sets differ across some comparisons. Astra results change depending on supplementary hints. Public benchmark settings differ from some paper runs. The six-system scaling analysis is exploratory and lacks independent validation. The history ablation and frozen-workspace studies use Astra and six tasks. The cross-task section contains only three first-run cases. None of this demonstrates open-ended recursive self-improvement, safe deployment or reliable transfer to unrestricted physical settings. The term recursive self-improvement carries science-fiction luggage. Here it means a bounded loop in which a fixed-weight model edits artifacts in its workspace, tries again and is measured under a fixed interaction budget. That is still useful. Physical attempts are expensive evidence. A dropped block, a jammed gripper and a bad approach angle should not vanish into the word failure. They should leave behind a trace that can be inspected, tested, transferred cautiously and rolled back. Keep the workspace. Then keep the receipt that explains what changed inside it.