Ai2 taught an 8B model to write cited reports in one pass

THE SIGNAL IN ONE SENTENCE

A citation can be correct and still tell a lie by stretching. A paper studies forty patients in one hospital. The generated report turns that finding into a claim about all patients. The source really is about the topic. The citation really does point to the paper. The sentence has still wandered beyond the evidence. That is the important problem behind AstaBrief, a new open-weights model from Ai2 for writing cited scientific reports. The model takes two things: a research question and excerpts retrieved from scientific papers. It then writes a structured report with inline references in one pass. It is based on Qwen3-8B, uses roughly eight billion parameters and is available under the Apache 2.0 license. Ai2 also released the supervised fine-tuning and preference datasets used to train it, plus an example workflow for generating reports from a local collection of PDFs. The plain signal is simple: an open model can now turn a prepared evidence packet into a useful first-pass literature report on infrastructure a lab controls. The citations make review possible. They do not make review optional. Ai2 introduced AstaBrief on October 2 as the new Fast mode inside Asta, its research platform. The existing Thinking mode uses a Claude-backed multi-step pipeline. AstaBrief collapses more of the writing process into a single model call. In Ai2's published measurements, the complete Fast-mode pipeline averaged 51.1 seconds per report. Thinking mode averaged 178.5 seconds. That makes the full workflow about 3.5 times faster. The company also describes the report-generation stage itself as nearly an order of magnitude faster than the proprietary systems it tracked. Those are different comparisons. The 3.5 times figure covers the full Asta pipeline, while the larger claim concerns a narrower stage. Both are developer-reported measurements. They are useful as engineering evidence, not as a universal speed guarantee for every machine, query or document collection. The model's small size changes who can use the idea. An eight-billion-parameter model is not tiny in the way a phone app is tiny. It still needs capable hardware and a serving stack. But it is small enough for a university, company or well-equipped lab to run on infrastructure it controls instead of sending unpublished questions and source material to a third-party model API. That can matter when a query exposes a planned experiment, an unfiled patent, confidential clinical work or a literature search that hints at a company's research direction. Local deployment does not automatically make the workflow private. Administrators still need access controls, encrypted storage, logging rules, retention limits and a plan for the source PDFs. A model behind a firewall can leak information through a careless interface just as efficiently as one in the cloud. Still, the option is real. The model card includes an inference example, a recommended prompt format and a maximum generation setting of 4,096 tokens. Ai2's accompanying code shows how to adapt the system to a user's own PDFs. The input matters as much as the model. AstaBrief does not begin with a question and magically search all science. Its intended job begins after a retrieval system has selected relevant excerpts. The prompt presents the query plus quoted passages identified by reference keys. The model writes from that packet and attaches those keys to its claims. That makes AstaBrief closer to a report writer at the end of a research pipeline than a complete research agent. If retrieval misses a decisive paper, the writer cannot rescue it. If an excerpt cuts away a limitation, the report may never see the limitation. If a retracted paper ranks highly, a neat citation can preserve the problem instead of fixing it. The one-pass design is the clever operational choice. Ai2's older report process retrieved literature, summarized snippets, clustered material into sections and generated each section separately. AstaBrief is trained to take the assembled question and references and write the report directly. Fewer intermediate stages mean less latency and fewer model calls. They also mean fewer places to inspect before the final report arrives. A multi-step system can expose its section plan, intermediate summaries and source groupings. A one-pass model compresses those decisions into its generation. That is convenient for a quick brief. It can be less convenient when a reviewer wants to know why one paper was emphasized and another was barely mentioned. The release is unusually transparent about how the behavior was taught. Ai2 began with real queries submitted to OpenScholar and Asta ScholarQA by people who opted into data sharing. It removed beta-tester and bot traffic, very short prompts, non-English and non-scientific requests, and prompts containing personal medical information. The filtering left about 90,000 research-focused queries. For supervised fine-tuning, a multi-step ScholarQA pipeline generated complete cited reports from those queries. The backing systems included Claude 3.5 Sonnet, Claude 3.7 Sonnet, o3, o4-mini and GPT-4.1. Ai2 says quality filtering produced 47,000 usable examples. A later citation-density filter kept 39,500 examples in the released SFT dataset. That distinction is easy to miss. Forty-seven thousand is the usable pool before the final citation-density threshold. Thirty-nine thousand five hundred is the released subset described on the dataset card. The team then used direct preference optimization, or DPO, to teach the model which of two reports was better. For each of roughly 6,600 prompts, one report came from the ScholarQA pipeline and another came from a different model working from the same retrieved excerpts. GPT-4.1 and DeepSeek-R1 judged the pairs. Ai2 kept examples only when both judges agreed, and reports 95 percent agreement between the model judges and human preferences in its alignment check. That gives the model a complicated family tree. The final weights are open, but much of the training behavior was distilled from proprietary systems. The released SFT and DPO datasets use the CC BY-NC 4.0 license, not the Apache license applied to the model. Their cards also warn that synthetic outputs remain subject to the terms of the third-party model providers that produced them. Anyone planning a commercial product should read those licenses and provider terms directly. Open weights do not make every ingredient equally permissive. Ai2 found that one blunt training-data filter helped more than several elaborate ones: citation density. The team measured what share of statements in a synthetic training report had at least one citation. It removed reports with a density below 0.25. According to the release, that simple filter improved the fine-tuned checkpoint more than combinations based on output length, retrieval relevance or citation diversity. This is an appealing result because it points toward a practical behavior rather than a bigger base model. Show the model more examples in which claims are consistently tied to sources, and it becomes better at producing traceable reports. Traceable is not the same as faithful. Ai2 evaluates citation precision, which asks whether a citation supports the attached claim, and citation recall, which asks whether report claims have supporting citations. The model card reports that AstaBrief scored 90.5 on citation precision and 78.2 on citation recall on its 100-question ScholarQA-CS2 test set, up from 76.2 and 64.6 for the base Qwen3-8B in the published table. The model's overall average in that table rose from 77.3 for Qwen3-8B to 87.0 for AstaBrief. Answer precision fell slightly, from 90.6 to 89.0, while coverage and citation metrics improved. Those scores describe one benchmark and Ai2's evaluation setup. They are not percentages of scientific truth. The stronger warning comes from Ai2 itself. A citation can support the subject of a sentence while the sentence broadens the source. A study about one population becomes a statement about everyone. A past observation becomes a timeless rule. A descriptive association becomes a recommendation for clinicians or policymakers. None of those shifts requires the model to fabricate a paper. It only has to sand away the boundaries. That is why a citation checker should ask more than whether the source is related. It should ask whether the sentence preserves the study's population, time, method, uncertainty, causal limits and strength of conclusion. For a working researcher, AstaBrief is most useful when treated as a drafting surface. Ask it for a map of competing methods. Use it to turn a carefully selected packet of papers into a first outline. Generate a surveillance brief for a field that changes every week. Compare how different retrieval sets change the synthesis. Run it locally when the question itself is sensitive. Then click the citations. For each important sentence, find the exact passage in the source. Check whether the model preserved sample size and population. Separate correlation from causation. Look for negative results, contradictory studies and limitations that did not fit into the retrieved excerpt. Confirm that the paper has not been corrected or retracted. The workflow can be useful even when the report is imperfect because it makes those checks faster. A blank page is expensive. A cited draft gives a reviewer a set of claims to attack. The attack is the job. Ai2's own evaluation limits deserve equal weight. Most of the training and evaluation described in the release was completed in 2025. The proprietary models used to generate data and serve as comparison points reflect that period. Ai2 says it did not rerun the full evaluation against the frontier systems available at release. The small human study covered fourteen questions supplied by three scientific researchers. They ranked reports from three systems with ties allowed. Ai2 reports that DR Tulu won overall preference, while two of the three researchers preferred AstaBrief on citation accuracy measures. Fourteen questions cannot settle the quality of scientific synthesis across medicine, climate science, materials, physics and the social sciences. It can show that the approach is worth testing. Early usage inside Asta is encouraging but similarly narrow. Ai2 says 374 people tried Fast mode. Of those users, 29.1 percent returned on at least two days. Twenty-three percent stayed with Fast mode for future report threads, and another 18 percent moved between Fast and Thinking modes. Positive feedback rates were close, at 84.2 percent for Fast and 85.2 percent for Thinking. These figures describe behavior among early users, not a controlled outcome study. Fast mode may be good enough for preliminary work, or users may simply like not waiting. The release cannot separate those explanations. There is also a privacy question in the training data. Ai2 says only queries from users who opted into data sharing were used, and that personal medical information was filtered. The public SFT dataset still contains real research questions paired with synthetic reports. Institutions should inspect the fields, license and curation process before adopting the data for another purpose. A useful deployment should build an evidence ledger around the model. Store the retrieval query, source version and exact excerpt shown to the model. Keep the model checkpoint, prompt and generation settings. Link every citation marker to the passage the reviewer saw. Flag claims that contain population-wide language, causal verbs, present-tense universals or recommendations. Require a human sign-off before a generated statement enters a paper, clinical decision, policy memo or public claim. The system should also make absence visible. If the retrieval set contains only five studies, say so. If every source comes from one discipline, country or time period, say so. If a claim relies on one paper, show that fragility. If the model used background knowledge outside the supplied evidence, label it separately. AstaBrief's released prompt does allow the model to add outside knowledge when it is confident and label the passage as model memory. For serious evidence work, many teams will want to disable that option or treat it as a bright warning rather than a source. The most useful thing about this release is not that an eight-billion-parameter model can produce handsome prose. Models have been producing handsome prose for years, sometimes to everyone's regret. It is that the complete package makes a specific workflow inspectable. There is a model card. There are open weights. There are training datasets. There is a prompt. There are benchmark tables. There is a local-PDF example. There are explicit caveats about old comparisons and evidentiary scope. That gives research groups something concrete to reproduce and improve. The next experiments should test claim scope directly, not only citation attachment. They should cover more fields and more languages. They should compare retrieval systems, measure omitted contradictory evidence and evaluate whether expert review gets faster without becoming lazier. They should test what happens when the source packet contains a bad paper, a retraction, a non-replicated result or two studies that genuinely disagree. An open scientific report model should not be judged by how confidently it finishes the literature review. It should be judged by how easily a careful reader can see where the evidence ends. AstaBrief makes that boundary easier to inspect than a source-free answer. Now the reviewer has to keep it from moving.