Beetle opened 330 language-model checkpoints. Make learning visible

THE SIGNAL IN ONE SENTENCE

Most model releases give you a graduation photo. Beetle gives you the school notebooks. The open-source research suite, described in an EMNLP paper published October 24, contains 285 bilingual language models and 45 monolingual baselines. The bilingual models span 21 first languages, with English used as the second language in the reported experiments. Each run comes with roughly 30 checkpoints showing the model at different moments in training. That makes 330 finished models, but the finished models are not really the point. Beetle is built to let researchers watch learning happen. The project varies when a model first encounters English, how much English it sees, whether exposure arrives steadily or in bursts, and how those choices interact with the amount of training data. The authors use five exposure schedules and three web-data scales: 100 million, 2 billion and 24 billion tokens. They also include a 100-million-token human-scale condition based on BabyBabelLM. All of the principal models use the same 125-million-parameter decoder architecture. That is small by current commercial standards. It is also useful. A research instrument does not need to write a screenplay, book a flight and redesign your database. It needs to let a scientist change one condition without accidentally changing six others. The plain signal is this: publish the learning trajectory, not only the final score. A training checkpoint is a saved state of a model at a particular moment. Compare checkpoints and you can ask when a grammatical pattern appeared, when knowledge from a first language helped or interfered with English, whether a behavior survived a change in exposure, and whether an apparent improvement was gradual or arrived after one specific phase. Without checkpoints, those questions become archaeology. Researchers see the finished weights and try to infer what happened inside a long training run. With checkpoints, the timeline becomes evidence. Beetle makes that timeline unusually dense. The paper says checkpointing follows a logarithmic schedule that restarts at each new exposure phase. Saves are frequent around transitions, when the learning dynamics may change quickly, and less frequent during quieter stretches. The released materials include model states in common formats plus tokenizers, evaluation scripts and the CSV files used to produce the paper's figures and tables. That is what open research looks like when the release is designed for questions rather than applause. The five exposure schedules are controlled approximations, not biographies of real language learners. One schedule presents the first language and English in a roughly even mixture from the beginning. Others delay English and introduce it gradually. One clusters English into intermittent bursts. The late schedule waits until halfway through training before beginning a transition toward English-heavy data. The researchers then compare how these schedules affect several kinds of behavior. For reading-time prediction, they use human eye-tracking data from the Multilingual Eye-movement Corpus. A language model assigns probabilities to words. A word the model finds unexpected has higher surprisal. Researchers can test whether that surprisal helps predict where human readers slow down, after accounting for ordinary factors such as word length and frequency. The authors report that staged and temporally structured exposure generally aligned better with second-language reading times than a balanced bilingual mixture. The largest gains appeared at smaller data scales and for more closely related language pairs. That sounds like a winner until the rest of the paper walks in. No single curriculum dominated every test. Balanced exposure produced the strongest grammaticality results on one evaluation. On learner-error discrimination, differences among curricula were smaller. The paper also reports that some 125-million-parameter Beetle models matched or exceeded much larger multilingual systems on cognitively oriented evaluations, but those comparisons are not fully controlled for architecture, training data, tokenizer or compute. That untidiness is useful. If one schedule wins every chart, the temptation is to turn a research finding into a universal recipe. Beetle instead shows that the answer depends on the question. A training history that better predicts reading time may not be the history that produces the strongest grammatical generalization. The model is not climbing one ladder called language ability. It is moving through a landscape of different behaviors. The checkpoints make that landscape inspectable. Suppose a team wants to know whether a bilingual model forgets its first language when English arrives. A final evaluation can say that performance declined. A checkpoint series can show when the decline began, whether it followed a sudden data shift, whether a later phase recovered the behavior and whether the same pattern appears across language pairs. Suppose another team wants to compare tokenizers. Beetle treats tokenization as an experimental variable and uses equal-compression tokenizers in the reported runs so both languages contribute comparable amounts of information per sequence. Future experiments can alter that choice while holding the model and exposure schedule steady. Suppose a third team wants to study low-resource languages. The framework can help organize controlled experiments, but the current release is not a victory lap for global coverage. The paper says the first-language sweep is typologically diverse while still providing limited coverage of lower-resourced languages. Larger-scale runs focus on nine first languages, and the experiments reported in the paper always point toward English as the second language. The direction matters. Learning English after Hindi is not the same research question as learning Hindi after English. A model trained on scraped web text is not a person learning through conversation, correction, desire, embarrassment, jokes, school, family and daily necessity. The authors say this plainly. Their curricula isolate exposure quantity and timing. They do not model social context, communicative goals, feedback, pragmatics or active language use. The paper also uses one random seed for its main configurations. Some comparisons with larger pretrained systems are not matched on data or compute. Neural signals such as N400 and P600 responses are outside the current evaluation. Those are not footnotes to hide. They are the border of the claim. Beetle's intended use is scientific study of bilingual language acquisition and reading dynamics. The authors explicitly say the models are not deployed second-language tutors and that the evaluation pipeline is not for placement decisions. That distinction should travel with every download. A small open model can generate text. That does not make it a fair judge of a student's proficiency, a teacher, a translator for consequential services or a psychological replica of a bilingual person. A reading-time correlation is evidence about one measurement under one setup. It is not evidence that the model understands what learning a language feels like. The practical opportunity is much better than pretending otherwise. Researchers can use Beetle to build comparison recipes. Start with one question. Does later English exposure improve alignment with reading behavior? Does a bursty schedule preserve more first-language structure? Does a different tokenizer change cross-language transfer? Does a continual-learning method reduce forgetting? Then freeze the parts that should not move. Record the exact data snapshot, language pair, tokenizer, model architecture, token budget, random seed, exposure schedule and evaluation version. Save checkpoints around every phase transition. Run more than one seed before treating a pattern as stable. Report the failures and the cases where different measures disagree. Make the evaluation table reconstructable. For every plotted point, publish the model checkpoint, input data, preprocessing, metric definition, analysis code and uncertainty. If a larger baseline uses a different tokenizer or training corpus, label the comparison as imperfect instead of letting a clean bar chart imply laboratory control that was never there. Then invite the language community in. A technically reproducible experiment can still ask the wrong question. Speakers, teachers, psycholinguists and local-language researchers should help choose phenomena, flag unnatural data, interpret code-switching and identify measurements that matter in the real language community. Open checkpoints make that collaboration easier because outside researchers can inspect a stage of learning instead of accepting a summary written after training ends. There is a broader lesson here for frontier model builders too. Commercial labs often publish a final model, a benchmark table and a system card. Checkpoint-rich releases are harder because they expose the path. That path can reveal regressions, contamination, forgotten capabilities, sudden behavioral shifts and results that depend on one convenient evaluation. Exactly. The training path is part of the evidence. Beetle is small enough to put that principle under a microscope. Its value is not that 330 compact models will replace the systems people use every day. Its value is that researchers can ask a narrow question, follow the change through training and hand the full trail to someone who disagrees. The finished model can pose for the photograph. Keep the notebooks open on the table.