Peking compressed the coding agent's memory, then paid for every missing detail

THE SIGNAL IN ONE SENTENCE

A coding agent can drown in its own receipts. Every file view, search result, stack trace and test log lands in the conversation, then gets carried into later model calls. The longer the job runs, the more of yesterday's terminal output the model rereads before deciding what to do next. Three researchers at Peking University, Zhensheng Zou, Guoqing Wang and Dan Hao, have tested a different memory arrangement. Their September 25 preprint calls it Latent Observations, Hard Actions, or LOHA. Older tool observations are compressed into continuous representations called soft tokens. The agent's own reasoning and tool calls remain ordinary text. The newest K observations also remain ordinary text, because a coding tool often needs the exact path, identifier or source line it just saw. A companion training method, Anchored Context Distillation, teaches the adapted model to read the compressed material while nudging its full-text behavior toward the unmodified base model. That is the clever part. The honest part is the bill. With the recent-text window set to three observations, the researchers report 43 percent less context per call for Qwen3-4B and 57 percent less for SWE-Master-4B-RL on SWE-bench Verified. Those agents also resolved fewer issues than their uncompressed bases: 12.1 percent versus 14.5 percent for Qwen3, and 21.8 percent versus 27.5 percent for SWE-Master. A larger recent window generally recovered some performance while giving back compression. In a single-run sweep, K equals eight reached 14.4 percent and 23.0 percent. That makes latent memory a dial, not a trophy. Turn it toward shorter contexts and the server can fit more work. Turn it toward visible text and the agent gets more exact evidence. The paper also reports a case where compression helps capability rather than merely price. Under a 32,768-token cap, Hard-Last-3 Qwen3 resolved 42 of a 199-instance subset, while the same adapted model using full text resolved 22. The compressed history kept more useful past information inside the limit. Yet each capped condition was run once, and the subset is not the complete benchmark. This is encouraging engineering evidence, not a production guarantee. The serving result is similarly specific. On one GPU with sixteen concurrent Qwen3 trajectories, LOHA completed 63.4 instances per GPU-hour compared with 33.4 for the adapted full-text agent. Median call latency fell from 5.07 seconds to 1.71 seconds. The current server also re-encoded old observations on every call, adding encoder work. A replay with per-observation caching estimated that most of that extra work could disappear, but an estimate is not a deployed measurement. Masking, the simpler approach of dropping older observations, reached 64.7 instances per GPU-hour in the same comparison. Compression preserved more historical information than masking, but it did not own the speed chart. Exact detail is where the method shows its seams. At the default sixteen-times compression ratio, a separate exact-recitation test recovered only 15.1 percent of 358 literals later quoted by actions. That is why LOHA keeps recent observations in text and allows old material to be fetched again through tools. A soft representation may preserve the gist of an earlier test failure while losing the punctuation needed for a precise replacement. It may remember that a configuration file mattered while blurring the value the agent must reproduce. The product implication is not to hide the original log. Keep an immutable human-readable trace outside the model context, record which observations were compressed, and let the agent or operator reopen the exact source. Otherwise a memory-saving feature becomes an audit-deleting feature. The paper also separates reading from behavior. A model can become better at reconstructing compressed text and worse at completing the coding task. In one small ablation, a student trained from a larger teacher exactly recited more literals but resolved fewer tasks than the self-anchored student. Anchoring matters because learning a new input representation can disturb the policy that chooses tools, edits code and decides when to stop. This is an important warning for every compressed-memory product. A good reconstruction score does not prove a good agent. The operational test is whether the complete trajectory reaches a correct patch without looping, quoting the wrong string or silently abandoning necessary evidence. For teams building coding agents, the practical design is a memory ladder. Keep the system instructions, task description, assistant actions and a recent evidence window as readable text. Compress older observations only after their exact source is safely stored. Attach an identifier to each compressed block so a reviewer can reopen it. Trigger retrieval when the agent needs a literal path, command, line or error message. Test several K values on the organization's own repositories because the right window depends on task shape, tool design and context limit. Report resolved tasks beside tokens, latency, throughput and retrieval count. A cheaper failed patch is still a failed patch. Human inspection needs its own evaluation. Can an engineer reconstruct why the agent changed a line? Can a security reviewer see whether a malicious instruction entered through a tool response before it was compressed? Can an incident investigator compare the latent view with the original observation? Can the system prove which exact text was available when a tool call was made? Soft tokens are not naturally readable, so the surrounding product has to supply this evidence. Compression may reduce what the model rereads. It should not reduce what the organization can audit. China matters here because the work comes from a Peking University team tackling a problem that every long-running agent faces: memory is becoming infrastructure. The result is not another giant model announcement. It is a systems proposal about where exact text belongs, which history can become approximate, and how training can add a new memory representation without replacing the agent's whole behavior. That is useful beyond coding. Research assistants, support agents and operations tools all accumulate observations. The boundary should remain the same: compress the material used for orientation, preserve or retrieve the material needed for exact action, and keep the original evidence available to people. The plain signal is pleasantly inconvenient. Latent memory can buy shorter contexts and more concurrent work. It can also lower task success and make the agent's past harder to inspect. The winning product will not compress everything and announce victory. It will expose the dial, preserve the receipts, measure the errors and know when one missing character is the whole job.