Working drawing 02 · The workings← The workings
How the model pages are made.
The tests run themselves. The overviews are written by an AI from the lab's records. This page says exactly what that means.
Measurements taken on an Apple M1 Max with 64 GB of unified memory.
Every number here comes from a locked dataset with a recorded grader. Where a measurement is not valid, it is marked rather than removed.
The tests run themselves
Every benchmark runs on an automated harness on one Mac Studio (M1 Max, 64 GB). The test sets never change once they're locked, the graders must pass their own checks before they score anything, and the settings are recorded with every run. Each finished run goes into the run log with ids for the test, its data, its grader and its settings. The scores on every model page are read from those records when the site is built. Nobody types them in.
The record behind each model
Every model family has a record in the lab's notebook: why it was tested, every file we held and how it was made, how it was served, each decision with its date, and what went wrong.
Most of these records were rebuilt on 23 October 2026 by Claude, an AI model, from the run log, the notes kept at the time and the working sessions where the testing was done. A script checked every quote against the session it came from. Where a reason had to be pieced together afterwards, the record says "reconstructed". Where something wasn't written down, it says "not on record" rather than guess. From now on, each session that tests a model updates its record before it ends.
The overviews
The overview at the top of each model family's page is written by Claude from that record, in plain words. It isn't Alan's writing, and he doesn't rewrite it. Every number in an overview comes from the record or the run log. Quotes are Alan's own words from the sessions, with spelling fixed and nothing else changed.
What we take out
The lab's record is private working material. Before any of it reaches this site, a script removes file paths, network details and references to private notes, names each machine by its hardware, and refuses to publish if anything it should have removed is still there.
What to keep in mind
- One Mac. Every score and speed here was measured on one Mac Studio. Other Macs will be faster or slower, and what fits your Mac shows memory, mostly as an estimate.
- An AI wrote the overviews. They're checked against the record, but they can still be wrong. If you spot something, write to hello@freshcontext.io.