Tencent let code agents branch instead of chasing one winner. Audit the merge

THE SIGNAL IN ONE SENTENCE

A code agent that keeps only its latest winner is easy to understand. It is also easy to fool. One experiment nudges a metric upward, so the system promotes it. A stranger idea dips at first, so the system throws it away. A third idea would have helped only if it had been combined with something discovered on another branch, but that branch is already gone. Tencent researchers want code evolution to remember more of its own history. Their new system, TGR-EvoDAG, represents automated code experiments as a directed acyclic graph. In plain language, it lets agents explore several paths, preserve what each path tried, revisit promising ancestors and merge ideas that appear complementary. The preprint was submitted October 23 and appeared in the October 26 arXiv listing. Its authors are affiliated with Tencent, Hong Kong Polytechnic University, McGill University and independent research in China. The target is generative recommendation, a family of systems that predicts the next item in a sequence by generating a semantic identifier. The specific training pipeline has three tightly coupled stages: a tokenizer turns items into identifiers, pre-training teaches a sequence model to predict them, and post-training aligns the model with recommendation objectives. Those stages make automated experimentation awkward. Change the final alignment method and only the last stage may need to run again. Change the tokenizer and the rest of the pipeline may need to be retrained around it. A search system that counts every experiment as equal can spend its budget on the wrong ideas. A system that judges every edit immediately can also kill a useful upstream change before downstream components have adapted. TGR-EvoDAG attacks both problems. Every experiment becomes a node with parents, a hypothesis, a code change, a codebase snapshot, related artifacts, validation feedback, lessons and consumed cost. New nodes can explore a different direction, mutate a local detail or merge multiple parent branches. Because a node is derived from evaluated ancestors without rewriting them, the history stays acyclic. Seven specialized agents execute a five-stage workflow. They choose an operator, propose and critique an architecture, code and review it, distill lessons and decide what to prune. The agents exchange structured records rather than a loose chat transcript. The cost planner is the practical bit. Each node records which pipeline stage it touches and the actual time spent there. Before another experiment runs, the selector estimates the GPU-hour cost and weighs it against the expected gain. Expensive upstream experiments can wait until cheaper downstream ideas mature. The other useful mechanism is downstream co-adaptation. If a tokenizer change initially hurts because later stages still expect the old representation, the graph can preserve that branch and schedule a child experiment that adapts the downstream model. A linear winner-take-all search would probably delete the evidence before that repair happens. The authors report a vivid example from their 39-round evolution run. A new tokenizer direction initially collapsed the test Hit@10 score to 0.0599. Diagnostics traced part of the failure to a semantic identifier collision spike. After normalization and parameter repair, the branch recovered. When the tokenizer was later merged into the main branch, performance dipped again to 0.0775. The system kept it alive, retuned the downstream loss and ultimately reached 0.0869. That sequence is more informative than a single winning score. It shows why lineage matters. A temporary regression can be either a bad idea or a good idea attached to an unprepared neighbor. The graph gives the team a way to tell those stories apart. On public Amazon recommendation datasets, the paper reports improvements over its strongest non-evolutionary baseline of 24.7 percent for Hit@10, 28.5 percent for Hit@20, 37.4 percent for NDCG@10 and 37.2 percent for NDCG@20. Agent4Rec simulated-user satisfaction improved 19.7 percent over that baseline. Those are author-reported offline and simulated results. They do not tell us whether a person enjoyed a real feed. The industrial evaluation gets closer. The researchers trained the evolved architecture and their APAO baseline from scratch on ten days of Tencent platform logs containing more than ten million interactions. They report a 12.16 percent relative improvement in offline Hit@1. Then they added the evolved model as one retrieval channel in Tencent's commercial media platform. The 14-day A/B test covered 5 percent of live traffic and more than one million users. The paper reports a statistically significant 0.40 percent increase in overall click-through rate, with a p-value of 0.019 after CUPED variance reduction and false-discovery-rate control. That is a real production signal. It is still one signal. A click is not satisfaction, trust, diversity, relevance over time or user welfare. The new channel may surface useful items the old system missed. It may also learn to win more attention. The paper says the channel contributed more than 10 percent unique exposures and had a channel-specific click-through rate about 13 percent above the system average. Those figures explain why a platform might deploy it. They do not close the evaluation. This is where the branch-and-merge idea needs its own audit trail. Start with lineage. Every deployed node should name all parent nodes, the hypothesis inherited from each one, the exact code diff and the model artifacts it produced. A merge should identify which behavior came from which branch. If the team cannot trace a failure back through the graph, the pretty structure is just a more complicated folder. Keep the dead branches too. The paper's graph marks falsified nodes. A production record should preserve why each one failed, which metric rejected it, what it cost and whether the failure appeared only after traffic exposure. Deleted experiments are where teams relearn expensive lessons. Budget needs two columns. GPU hours are one. User exposure is the other. An offline experiment spends compute. An online experiment spends a slice of people's attention. Record traffic percentage, duration, eligibility rules, guardrails and stopping conditions beside training cost. A branch that is cheap to train can still be expensive to test if it produces repetitive, unsafe or manipulative recommendations. Merges deserve a conflict review. Combining two improvements can break interfaces, invalidate prior measurements or create an interaction neither parent displayed. Before promotion, test the merged model against each parent, the current production baseline and a clean holdout. Look for gains in the target metric and regressions in latency, diversity, creator distribution, complaint rates, session quality and vulnerable-user outcomes. The holdout should survive the celebration. If every user eventually receives the evolved system, the team loses its view of long-term effects. Keep a stable comparison group where policy allows. Recheck whether the click gain persists, whether people return voluntarily and whether the recommendation mix narrows over weeks rather than days. Rollback also has to be a graph operation. A team should be able to retreat from the merged node to a known parent without reconstructing the entire pipeline under pressure. Store compatible checkpoints, data versions, feature contracts and deployment configuration. Test that rollback before the experiment, not after a dashboard turns red. The system's seven agents do not remove human responsibility. They move it. People may write less experimental code by hand. They now need to define metrics, constrain search, inspect diagnostic claims, approve traffic exposure and decide whether a click gain is worth the trade. The authors say TGR-EvoDAG has become the evolution engine inside Tencent's internal Uppimo platform. That makes governance part of the product, not a paper appendix. The research is promising, but its limits are clear. The results come from the system's authors. The industrial code, logs, internal platform and full deployment environment are not available for independent reproduction from the paper alone. The public benchmark comparison uses the authors' standardized platform and choices. The 14-day online test demonstrates a short-term CTR change for one retrieval channel, not a universal advantage for autonomous code evolution. The reported statistical controls reduce one kind of uncertainty. They do not measure unreported outcomes. That distinction matters beyond recommendation. Any multi-stage AI pipeline can inherit the same temptation: preserve many ideas, merge the good ones and let agents search the space faster than people can follow. The answer is not to force every search into one timid line. It is to make the graph legible. Let agents branch. Let them keep a promising failure alive. Let them merge discoveries humans might not have paired. But require the winning node to arrive with parents, costs, conflicts, holdout results and a tested route home. The merge is where the cleverness becomes a product. It is also where the audit should get serious.