Working drawing 02 · The workings← Benchmarks
The workings.
How the numbers on this site are made, and every record behind them. Read the methodology first if you want to know why a number can be trusted.
- Suites
- 17
- Run log entries
- 682
- Experiments
- 10
Measurements taken on an Apple M1 Max with 64 GB of unified memory.
Every number here comes from a locked dataset with a recorded grader. Where a measurement is not valid, it is marked rather than removed.
- Methodology
The rules: tests that never change once locked, graders that must pass their own checks before they score anything, and what gets a run flagged.
- Suites
The 17 tests. Each asks a model one question, with how it’s graded and its known weak spots.
- Run log
Every run, 682 of them, flagged ones included, with the ids that show which test, data and settings produced it.
- Experiments
10 questions put to the hardware rather than the models: memory, speed, and the software that runs them.
- Cloud
The same tests run on 3 models in Ollama’s cloud, for comparison, with the caveats.
- Data
Every record behind these pages, to download as JSON. Free to reuse with credit (CC BY 4.0).