Model · measured only← Models
spark-x2.6-4b-q4km-np1-c64k-b11846
15 recorded runs. This model has no written entry yet.
- Scored runs
- 15
- Run log entries
- 15
- File
- 3.5 GB
- Needs
- 7.8 GB measured, 64K context
- Smallest Mac
- 16 GB
spark-x2.6-4b-q4km-np1-c64k-b11846
Every result below is as recorded. Without a written entry there is no stated reason for the test and no verdict.
Of record
The measurement of record for this model.
The run’s own note: Single slot at 64K (benchmark rule), beside the full fleet. Vendor Q4_K_M is a mixed quant (BF16 token embeddings, Q8_0 attn-output and output head). shapegen and sitebuild excluded.
- Server
- 1 slot, 64K of context per request, KV cache f16 (default), flash attention on, 1 suite at a time
- Runtime
- llama.cpp b11846 (1a24df163), plain upstream
- Speed
- decode 57.8 tokens/s, prefill 465.4 tokens/s (435.5 on prompts over 16K), medians. Single stream, full fleet resident (contended).
- Peak memory
- 7.8 GB resident, with 64K of context allocated
- Weights
- 3.5 GB file · sha256
91cb2811aebb47ac… - Coherence check
- passed
- Left out by plan
- shapegen, sitebuild
- Machine
- Mac Studio (M1 Max, 64 GB)
- Released
- 24 October 2026, bundle
2026-10-24-4174a46f
Memory by context
The same file and server, started at each context size in turn: what it held in memory and how fast it wrote on a short prompt. It measures memory and serving, not how well the model uses a long context. Released 24 October 2026, bundle 2026-10-24-743cfeb3.
The ladder’s own note: Context ladder 8K to 256K (native window 1M; the ladder tops out at 262144), single slot, f16 KV, beside the full fleet. Decode on a trivial prompt per rung; measures allocation and serving, not long-context quality.
| Context | Resident memory | Swap in use, whole Mac | Decode | Load |
|---|---|---|---|---|
| 8K | 3.81 GB | 11.76 GB | 58.1 tokens/s | 3 s |
| 16K | 4.09 GB | 11.76 GB | 58.3 tokens/s | 3 s |
| 32K | 4.66 GB | 11.76 GB | 58.2 tokens/s | 3 s |
| 64K | 5.78 GB | 11.76 GB | 58.2 tokens/s | 3 s |
| 128K | 8.04 GB | 11.76 GB | 58.1 tokens/s | 3 s |
| 256K | 12.54 GB | 11.76 GB | 58.2 tokens/s | 3 s |
| Suite | Score | 95% interval | Passed | Runtime · profile | Recorded |
|---|---|---|---|---|---|
| chainwalk | 50.0% | 9.5–90.5 | Passed 1/2 | llama.cppextraction | 2026-10-24 |
| contract | 29.2% | 18.2–43.2 | Passed 14/48 | llama.cppextraction | 2026-10-24 |
| enrichment | 85.2% | 73.4–92.3 | Passed 46/54 | llama.cppextraction | 2026-10-24 |
| epistemics | 95.8% | 86.0–98.8 | Passed 46/48 | llama.cppextraction | 2026-10-24 |
| grounding | 72.0% | 63.6–79.1 | Passed 90/125 | llama.cppextraction | 2026-10-24 |
| needle | 87.3% | 79.8–92.3 | Passed 96/110 | llama.cppextraction | 2026-10-24 |
| nullschema | 63.9% | 52.4–74.0 | Passed 46/72 | llama.cppextraction | 2026-10-24 |
| screening | 98.3% | 91.1–99.7 | Passed 59/60 | llama.cppextraction | 2026-10-24 |
| synthesis | 81.7% | 70.1–89.4 | Passed 49/60 | llama.cppextraction | 2026-10-24 |
| tagger | 87.9% | 83.2–91.5 | Passed 211/240 | llama.cppextraction | 2026-10-24 |
| tagger_nullable | 71.7% | 65.7–77.0 | Passed 172/240 | llama.cppextraction | 2026-10-24 |
| toolcall | 0.0% | -0.0–6.0 | Passed 0/60 | llama.cppextraction | 2026-10-24 |
| tools | 86.7% | 62.1–96.3 | Passed 13/15 | llama.cppextraction | 2026-10-24 |
| toolscale | 95.0% | 83.5–98.6 | Passed 38/40 | llama.cppextraction | 2026-10-24 |
| worldview | 40.0% | 26.3–55.4 | Passed 16/40 | llama.cppextraction | 2026-10-24 |
Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.
Every run log entry for this model (15)
| Finished | Suite | Runtime · profile | Score | Coverage | Grader · dataset · config |
|---|---|---|---|---|---|
| chainwalkchainwalk/3 | llama.cppextraction | 50.0% | Coverage 2/2 | g f4bfcf6f040b0978d 129a9e74408c8919c 484355bded7c5ccf | |
| contractcontract/2 | llama.cppextraction | 29.2% | Coverage 48/48 | g cc2d23c5bae45474d 1fe53d30cb5480f6c a4f6da9cb513bd95 | |
| screeningscreening/1 | llama.cppextraction | 98.3% | Coverage 60/60 | g 0661b49a10be2d2ad 64c97ca2269c29afc 576a7775cfad15c8 | |
| toolstools/1 | llama.cppextraction | 86.7% | Coverage 15/15 | g 2763675a5c3da564d 33a44812f0ad4c39c 2aaa6d3e20791385 | |
| toolscaletoolscale/1 | llama.cppextraction | 95.0% | Coverage 40/40 | g a2bc3751abfa28e0d b24abb108c409178c b55c738a4fe04d02 | |
| toolcalltoolcall/1 | llama.cppextraction | 0.0% | Coverage 60/60 | g 804e8fc9457649d8d 10af88c170513155c 53999f02f507d01c | |
| nullschemanullschema/1 | llama.cppextraction | 63.9% | Coverage 72/72 | g d7448e0f5fc3f9afd 3b91280bb40372efc 9f423724838731e4 | |
| worldviewworldview/2 | llama.cppextraction | 40.0% | Coverage 40/40 | g 9acfba1b704feb28d e97835de9e6fc227c d61317fe3d8dbd88 | |
| synthesissynthesis/2 | llama.cppextraction | 81.7% | Coverage 60/60 | g 4156dd3c13361e8bd 267d1954e225c5b6c d06c8b2257d089b2 | |
| epistemicsepistemics/4 | llama.cppextraction | 95.8% | Coverage 48/48 | g acf84972a0ad5f38d 4bce453fd376553cc 1174e5574995ebe3 | |
| enrichmentenrichment/3 | llama.cppextraction | 85.2% | Coverage 54/54 | g b51222c2ca38cfdfd 75361864baa727abc e19077cf9cdc909a | |
| groundinggrounding/1 | llama.cppextraction | 72.0% | Coverage 125/125 | g 30b933dcc6a7e2d4d dcb396faf8b83b0ac b09bb0071f669f4f | |
| needleneedle/1 | llama.cppextraction | 87.3% | Coverage 110/110 | g c38dec33aa1c86c2d 5c5199b9eee8a595c b9316008e0cd79fa | |
| tagger_nullabletagger-nullable/1 | llama.cppextraction | 71.7% | Coverage 240/240 | g 04e875431e26c4b5d db3addf23a2bc4cec c3ded5d798202b34 | |
| taggertagger/2 | llama.cppextraction | 87.9% | Coverage 240/240 | g 63e04d139a0e5ed3d 7ef587bf33e5788ac ce362973f5721739 |