EVERY SIGNAL, SORTED.

Google Research, Continual Learning, Language Models, Model Memory, Catastrophic Forgetting, Mixture of Experts, Knowledge Distillation, Synthetic Data, Reinforcement Learning, Privacy, Model Evaluation and Rollback

Google gave language models a sleep cycle. Audit what the model remembers

A research framework periodically moves temporary context into model parameters and rehearses synthetic examples. That could make models less forgetful. It could also turn a bad memory into a durable one.