THE COLLECTION
EVERY SIGNAL, SORTED.
Search the whole shelf by topic, tool, term, or plain old curiosity.
AI, Mathematics and Research Culture
Twenty-five Fields Medalists say AI is optimizing the wrong math problem
A declaration from leading mathematicians says famous problems are being turned into AI scoreboards. A correct endpoint matters, but so do the ideas, credit, teaching, and questions that make it mathematics.
Scientific AI and Evaluation
The best AI research assistant still misses nearly four points out of ten
SciDocBench tests the complete scientific reading workflow, and its strongest evaluated model scored 62.6 while struggling to keep papers, figures, equations, code, and evidence connected.
Scientific AI and Evaluation
The chemistry model may be remembering the answer, digit for digit
German researchers tested 22 frontier models on molecular-property benchmarks and found that some impressive scores were partly built from retrieving published measurements.
Local AI and Benchmark Reliability
The optimization worked because the original software picked the wrong kernel
A 20 percent Qwen speedup looked like a clever model trick. A fairer comparison revealed that most of the gain came from avoiding one badly chosen GPU execution path.
Engineering Agents and Physical Verification
AI circuit design finally got a test that plugs the answer into physics
EEBench makes models submit circuit designs that survive simulation, hidden tolerance corners, and a component budget. Confident prose receives no points.
AI Evaluation and Reliability
The benchmark judge changed its mind overnight
A preregistered audit sent identical work to language-model judges and received materially different rankings. The ruler was moving while everyone argued about the measurement.
Open Models and Evaluation
K2 Horizon shows what radical AI openness actually looks like
The model family arrives with weights, code, checkpoints, logs, and a wonderfully awkward disclosure about benchmark cheating. The mistakes may be the most valuable part.
Research and Evaluation
The benchmark says safety. The questions may be measuring something else.
Ai2’s open BenchMIRT method looks beneath a single score and asks whether an evaluation is testing safety, reasoning, or a muddy mixture of both.