Model · measured only← Models
flint-q4_1-np1-c64k-b11846
15 recorded runs. This model has no written entry yet.
- Scored runs
- 15
- Run log entries
- 15
- File
- 26.0 GB
- Needs
- 26.4 GB measured, 64K context
- Smallest Mac
- 36 GB
RecordIdentifier
flint-q4_1-np1-c64k-b11846
Every result below is as recorded. Without a written entry there is no stated reason for the test and no verdict.
SetupHow it was run
Of record
The measurement of record for this model.
The run’s own note: Single slot at 64K (benchmark rule). shapegen and sitebuild excluded: unfinished suite / harness dataset fault.
- Server
- 1 slot, 64K of context per request, KV cache f16 (default), flash attention on, 1 suite at a time
- Runtime
- llama.cpp b11846 (1a24df163), plain upstream
- Speed
- decode 74.2 tokens/s, prefill 345.2 tokens/s (444.7 on prompts over 16K), medians. Single stream, fleet resident except the vacated play seat (contended).
- Peak memory
- 26.4 GB resident, with 64K of context allocated
- Weights
- 26.0 GB file · sha256
43c07baccdba42eb… - Coherence check
- passed
- Left out by plan
- shapegen, sitebuild
- Machine
- Mac Studio (M1 Max, 64 GB)
- Released
- 24 October 2026, bundle
2026-10-24-746b3da7
Recorded runs15 scored · 95% intervals
| Suite | Score | 95% interval | Passed | Runtime · profile | Recorded |
|---|---|---|---|---|---|
| chainwalk | 50.0% | 9.5–90.5 | Passed 1/2 | llama.cppextraction | 2026-10-24 |
| contract | 60.4% | 46.3–73.0 | Passed 29/48 | llama.cppextraction | 2026-10-24 |
| enrichment | 77.8% | 65.1–86.8 | Passed 42/54 | llama.cppextraction | 2026-10-24 |
| epistemics | 95.8% | 86.0–98.8 | Passed 46/48 | llama.cppextraction | 2026-10-24 |
| grounding | 77.6% | 69.5–84.0 | Passed 97/125 | llama.cppextraction | 2026-10-24 |
| needle | 60.0% | 50.7–68.7 | Passed 66/110 | llama.cppextraction | 2026-10-24 |
| nullschema | 70.8% | 59.5–80.1 | Passed 51/72 | llama.cppextraction | 2026-10-24 |
| screening | 98.3% | 91.1–99.7 | Passed 59/60 | llama.cppextraction | 2026-10-24 |
| synthesis | 78.3% | 66.4–86.9 | Passed 47/60 | llama.cppextraction | 2026-10-24 |
| tagger | 82.1% | 76.7–86.4 | Passed 197/240 | llama.cppextraction | 2026-10-24 |
| tagger_nullable | 60.8% | 54.5–66.8 | Passed 146/240 | llama.cppextraction | 2026-10-24 |
| toolcall | 55.0% | 42.5–66.9 | Passed 33/60 | llama.cppextraction | 2026-10-24 |
| tools | 100.0% | 79.6–100.0 | Passed 15/15 | llama.cppextraction | 2026-10-24 |
| toolscale | 95.0% | 83.5–98.6 | Passed 38/40 | llama.cppextraction | 2026-10-24 |
| worldview | 75.0% | 59.8–85.8 | Passed 30/40 | llama.cppextraction | 2026-10-24 |
Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.
Every run log entry for this model (15)
| Finished | Suite | Runtime · profile | Score | Coverage | Grader · dataset · config |
|---|---|---|---|---|---|
| chainwalkchainwalk/3 | llama.cppextraction | 50.0% | Coverage 2/2 | g f4bfcf6f040b0978d 129a9e74408c8919c 88345f03f0e327c4 | |
| contractcontract/2 | llama.cppextraction | 60.4% | Coverage 48/48 | g cc2d23c5bae45474d 1fe53d30cb5480f6c 43204cdedbb980ca | |
| screeningscreening/1 | llama.cppextraction | 98.3% | Coverage 60/60 | g 0661b49a10be2d2ad 64c97ca2269c29afc 3178f115c8d05533 | |
| toolstools/1 | llama.cppextraction | 100.0% | Coverage 15/15 | g 2763675a5c3da564d 33a44812f0ad4c39c c2432e96b7bff4bf | |
| toolscaletoolscale/1 | llama.cppextraction | 95.0% | Coverage 40/40 | g a2bc3751abfa28e0d b24abb108c409178c ca6e6648773e72ef | |
| toolcalltoolcall/1 | llama.cppextraction | 55.0% | Coverage 60/60 | g 804e8fc9457649d8d 10af88c170513155c 956656f55d16d854 | |
| nullschemanullschema/1 | llama.cppextraction | 70.8% | Coverage 72/72 | g d7448e0f5fc3f9afd 3b91280bb40372efc 6f004eb5725740b2 | |
| worldviewworldview/2 | llama.cppextraction | 75.0% | Coverage 40/40 | g 9acfba1b704feb28d e97835de9e6fc227c 6bd294468f0eb16d | |
| synthesissynthesis/2 | llama.cppextraction | 78.3% | Coverage 60/60 | g 4156dd3c13361e8bd 267d1954e225c5b6c cfd189f0cc9137d8 | |
| epistemicsepistemics/4 | llama.cppextraction | 95.8% | Coverage 48/48 | g acf84972a0ad5f38d 4bce453fd376553cc ab518cd898f25ee8 | |
| enrichmentenrichment/3 | llama.cppextraction | 77.8% | Coverage 54/54 | g b51222c2ca38cfdfd 75361864baa727abc 6a8259b35c6b5b0d | |
| groundinggrounding/1 | llama.cppextraction | 77.6% | Coverage 125/125 | g 30b933dcc6a7e2d4d dcb396faf8b83b0ac abfb64f4eb2cecaf | |
| needleneedle/1 | llama.cppextraction | 60.0% | Coverage 110/110 | g c38dec33aa1c86c2d 5c5199b9eee8a595c 121389ab32a52e87 | |
| tagger_nullabletagger-nullable/1 | llama.cppextraction | 60.8% | Coverage 240/240 | g 04e875431e26c4b5d db3addf23a2bc4cec d2430ea674cbd7d9 | |
| taggertagger/2 | llama.cppextraction | 82.1% | Coverage 240/240 | g 63e04d139a0e5ed3d 7ef587bf33e5788ac abb95e6a254a1a93 |