Kyutai put memory in the robot's action expert. The real room still gets a vote

THE SIGNAL IN ONE SENTENCE

A robot arm can look decisive while being completely stuck. It reaches toward a block. The next camera frame makes the old pose look attractive, so the arm retreats. The following frame favors the first pose again. Back and forth it goes, a very expensive metronome with a gripper. The problem is not necessarily that the robot cannot see. It may be that it cannot cheaply remember what it just saw itself doing. Kyutai, the Paris-based nonprofit AI laboratory, has released REKI, a 2.4-billion-parameter vision-language-action model designed to carry a short record of its recent past inside the part of the system that predicts actions. The model uses a vision-language backbone to encode camera views and an instruction, then passes that context to a smaller action expert that predicts a one-second chunk of robot motion. The memory lives in thirty-two learned register tokens placed before the action tokens for each frame. Those registers summarize the current visual context, persist in a cache and can be read when the next action chunk is generated. In the released setup, the model trains on windows of three frames taken one second apart. At inference, it keeps registers from the two preceding frames as a rolling memory. That is the clever part. The practical part begins when the robot leaves the benchmark. The plain signal is this: remembering the previous pose can stop a robot from wobbling between two choices. It cannot tell you whether the next choice is safe in your room. REKI stands for Registering Knowledge in vision-language-action models. The work comes from researchers at Kyutai, Inria and École nationale des ponts et chaussées. The paper appeared on October 15, and Kyutai published the project materials on October 16. The model combines a Qwen3.5-2B vision-language backbone with a roughly 200-million-parameter flow-matching action expert. The backbone reads the instruction and camera views. The action expert uses cross-attention to that representation while robot state and flow time condition the action prediction. The registers sit inside the expert rather than forcing the much larger visual backbone to re-encode several old images for every decision. They are computed once for each frame, cached and reused across denoising steps. Because the registers do not depend on the predicted actions, the system can share them across steps without carrying an entire separate memory model. Kyutai reports that reading the cached registers adds 0.1 milliseconds to a 21.8 millisecond action-chunk prediction on one H100 GPU when the model receives two 256-by-256 camera images. The cache adds about two mebibytes of GPU memory for each past frame. Those numbers make the memory look almost free. They are also measurements from a specific compiled path on a high-end data-center accelerator. A robot controller, workstation and embedded computer will not inherit the same timing by osmosis. The team compares REKI with several released robot policies and reports the fastest compiled action-chunk time in that group. Some competing implementations were compiled by the researchers for the comparison. That is useful engineering evidence, but it is not a universal ranking of deployed robot systems. Sensor capture, control frequency, network transport, safety checks and the physical robot all add time outside the model call. Memory appears to help the behavior the researchers targeted. On RoboTwin Clean, Kyutai reports oscillation in 18.6 percent of episodes without memory and 4.1 percent with a three-frame memory window. The team attributes the difference to perceptual aliasing: two poses can each resemble a demonstration that continues toward the other, so a memoryless policy switches directions at every new chunk. A short history tells the model which side of the motion it just came from. That is a plausible and useful explanation. It does not mean the memory has learned a general understanding of cause, intention or task progress. The register tokens summarize recent observations. They do not maintain a symbolic plan, inspect a collision map or remember an entire afternoon. The rolling window is short. If the relevant event happened several seconds ago, disappeared behind an object or involved a force the cameras did not capture, the memory may not contain the information the robot needs. The benchmark results are strong within their frames. The released repository reports 97.6 percent success across four standard LIBERO suites and 90.4 percent across LIBERO-Plus, which perturbs camera viewpoint, initial robot state, language, lighting, background, sensor noise and object layout. On RoboTwin Clean2Rand, where a model tuned on clean scenes faces randomized conditions, Kyutai reports a mean success rate of 65.8 percent. On RoboCasa365, it reports 38.4 percent across fifty target kitchen tasks. The numbers are not interchangeable. Each benchmark uses different robots, tasks, training recipes and evaluation conditions. A 90.4 on one simulation suite does not cancel a 38.4 on another. More importantly, a benchmark success rate is not the probability that a robot will complete an arbitrary household task safely. The LIBERO model card says the released checkpoint is evaluated in simulation only and has no real-robot or safety evaluation. Kyutai's broader project page includes real-robot demonstrations and says memory improved success in simulation and on real robots. Those claims belong together with the boundary: demonstrations show that the architecture can move physical hardware, while the released benchmark card does not establish general real-world reliability or safety. A real room is rude in ways a benchmark rarely has time to be. Sunlight moves. A reflective bowl changes appearance. A sleeve enters the workspace. The table shifts. Two objects touch. A drawer sticks. A person gives a vague instruction, then changes their mind. The gripper nudges something that was not part of the plan. A pet performs an unscheduled audit. The useful deployment test is therefore not another clean success reel. It is a recovery ledger. Put the robot in the exact room where it will work. Vary lighting, clutter, camera position, object texture and starting pose. Interrupt actions. Move an object after the plan begins. Partially block the camera. Give instructions with ambiguous references. Measure how often the policy stops, retries, oscillates, drops an object, contacts the environment or needs human intervention. Record near misses as carefully as successful completions. A robot that eventually places the bowl after bumping a glass is not a clean success. A robot that completes nine trials quickly and makes one hazardous move may be less useful than a slower system that recognizes uncertainty and stops. Memory itself needs a failure test. Clear the register cache in the middle of an episode and see how behavior changes. Feed stale frames. Shift the camera while the cache still represents the earlier viewpoint. Start a new task without a proper reset. The public example explicitly clears the policy memory at every episode boundary. A production system should prove that the reset happens, that cache state cannot leak across users or tasks and that a corrupted history fails safely. The operator needs a physical stop control outside the model. No learned policy should decide whether its own emergency stop is valid. Workspace limits, collision detection, force thresholds, speed caps and human presence sensors belong in independent safety layers. The model can propose motion. A controller with simpler, testable rules should decide whether that motion may reach the motors. REKI's public release is unusually useful for research. Kyutai has published the training, evaluation, data-processing and deployment code, along with pretrained and task-specific weights. The repository documents recipes for LIBERO, RoboTwin and RoboCasa checkpoints and includes a real-robot policy server. The laboratory also says pretraining used public data only: robot demonstrations, simulation, egocentric human manipulation and vision-language datasets. It trained for 200,000 steps on thirty-two H100 GPUs. Public inputs and code make the work more inspectable. They do not make reproduction effortless. Dataset versions, preprocessing, hardware, random variation and the cost of a large training run still matter. The release terms matter too. The repository code includes permissive licenses, while the published model weights use CC BY-NC-SA 4.0. That allows research and noncommercial reuse under attribution and share-alike conditions, but it is not a general commercial deployment license. Teams should keep the model receipt attached to the robot. Record the exact checkpoint, code revision, camera calibration, normalization statistics, task instruction, robot embodiment, control rate and memory-window setting. The LIBERO checkpoint is tuned for a Franka Panda in simulation. The RoboTwin checkpoint targets a bimanual ALOHA setup. The RoboCasa model targets a mobile PandaOmron configuration. These are not decorative metadata. They define the body whose actions the model learned to predict. Changing the body changes the problem. A different gripper, joint range, camera mount or action representation can invalidate the behavior. Fine-tuning on a handful of demonstrations may produce a compelling demo while leaving large gaps around unusual objects and recovery. Kyutai shows physical rollouts after fine-tuning on fifty demonstrations per task, which is promising evidence for adaptation, not permission to skip a risk assessment. The strongest pilot begins with a narrow task and an intentionally annoying room. Choose one manipulation job with a clear success definition and a safe failure state. Establish a baseline without memory. Add the short register window. Run repeated trials across ordinary variation and designed disturbances. Compare task success, oscillation, time, contacts, stops, interventions and recovery. Then invite someone who did not build the model to break it. Independent operators notice different ambiguities. Safety reviewers ask what happens when the sensor lies. Workers know which objects get stacked badly and which corners attract clutter. People who share the space can say whether the robot's motion is understandable or unsettling. The point of robot memory is not to make the machine look more confident. It is to give the policy enough recent context to make fewer stupid reversals. REKI offers a clean, efficient mechanism for that. Thirty-two registers can carry a little past into the next action. The release makes the idea unusually inspectable and gives researchers a credible platform to test. But a short memory is not a conscience, a safety case or a map of the room. The robot can remember where its arm was one second ago. The people around it still decide where that arm is allowed to go next.