Model · measured only← Models
spark-x2.5-4b-q4km-np1-c64k-b11846
15 recorded runs. This model has no written entry yet.
- Scored runs
- 15
- Run log entries
- 16
- File
- 2.6 GB
- Needs
- 7.0 GB measured, 64K context
- Smallest Mac
- 16 GB
spark-x2.5-4b-q4km-np1-c64k-b11846
Every result below is as recorded. Without a written entry there is no stated reason for the test and no verdict.
Of record
The measurement of record for this model.
The run’s own note: Single slot at 64K (benchmark rule), beside the full fleet: the training comparison for Spark-X2.6-4B. Weights re-downloaded and pinned to a revision; the vendor Q4_K_M here is a plain Q4_K/Q6_K mix. On the five suites run on 2026-09-10, 270 of 272 answers are byte-identical to that run. shapegen and sitebuild excluded.
- Server
- 1 slot, 64K of context per request, KV cache f16 (default), flash attention on, 1 suite at a time
- Runtime
- llama.cpp b11846 (1a24df163), plain upstream
- Speed
- decode 58.4 tokens/s, prefill 453.2 tokens/s (433.2 on prompts over 16K), medians. Single stream, full fleet resident (contended).
- Peak memory
- 7 GB resident, with 64K of context allocated
- Weights
- 2.6 GB file · sha256
adfcfa19a4ed6a59… - Coherence check
- passed
- Left out by plan
- shapegen, sitebuild
- Machine
- Mac Studio (M1 Max, 64 GB)
- Released
- 24 October 2026, bundle
2026-10-24-0b6bf0e3
Memory by context
The same file and server, started at each context size in turn: what it held in memory and how fast it wrote on a short prompt. It measures memory and serving, not how well the model uses a long context. Released 24 October 2026, bundle 2026-10-24-eaf7f863.
The ladder’s own note: Context ladder 8K to 256K (native window 1M; the ladder tops out at 262144), single slot, f16 KV, beside the full fleet. Decode on a trivial prompt per rung; measures allocation and serving, not long-context quality.
| Context | Resident memory | Swap in use, whole Mac | Decode | Load |
|---|---|---|---|---|
| 8K | 2.93 GB | 10.02 GB | 59.7 tokens/s | 3 s |
| 16K | 3.22 GB | 10.02 GB | 59.7 tokens/s | 3 s |
| 32K | 3.78 GB | 10.02 GB | 59.6 tokens/s | 3 s |
| 64K | 4.91 GB | 10.02 GB | 59.5 tokens/s | 3 s |
| 128K | 7.16 GB | 10.02 GB | 59.7 tokens/s | 3 s |
| 256K | 11.62 GB | 10.02 GB | 59.5 tokens/s | 6 s |
| Suite | Score | 95% interval | Passed | Runtime · profile | Recorded |
|---|---|---|---|---|---|
| chainwalk | 50.0% | 9.5–90.5 | Passed 1/2 | llama.cppextraction | 2026-10-24 |
| contract | 25.0% | 14.9–38.8 | Passed 12/48 | llama.cppextraction | 2026-10-24 |
| enrichment | 77.8% | 65.1–86.8 | Passed 42/54 | llama.cppextraction | 2026-10-24 |
| epistemics | 72.9% | 59.0–83.4 | Passed 35/48 | llama.cppextraction | 2026-10-24 |
| grounding | 72.8% | 64.4–79.8 | Passed 91/125 | llama.cppextraction | 2026-10-24 |
| needle | 63.6% | 54.3–72.0 | Passed 70/110 | llama.cppextraction | 2026-10-24 |
| nullschema | 48.6% | 37.4–59.9 | Passed 35/72 | llama.cppextraction | 2026-10-24 |
| screening | 96.7% | 88.6–99.1 | Passed 58/60 | llama.cppextraction | 2026-10-24 |
| synthesis | 28.3% | 18.5–40.8 | Passed 17/60 | llama.cppextraction | 2026-10-24 |
| tagger | 87.5% | 82.7–91.1 | Passed 210/240 | llama.cppextraction | 2026-10-24 |
| tagger_nullable | 88.8% | 84.1–92.2 | Passed 213/240 | llama.cppextraction | 2026-10-24 |
| toolcall | 0.0% | -0.0–6.0 | Passed 0/60 | llama.cppextraction | 2026-10-24 |
| tools | 80.0% | 54.8–93.0 | Passed 12/15 | llama.cppextraction | 2026-10-24 |
| toolscale | 95.0% | 83.5–98.6 | Passed 38/40 | llama.cppextraction | 2026-10-24 |
| worldview | 20.0% | 10.5–34.8 | Passed 8/40 | llama.cppextraction | 2026-10-24 |
Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.
Every run log entry for this model (16)
| Finished | Suite | Runtime · profile | Score | Coverage | Grader · dataset · config |
|---|---|---|---|---|---|
| toolcall2toolcall/2 | llama.cppextraction | n/a | Coverage 120/120 | g 4c7867512a0a37b1d 2ecae6a15d157146c 2cd897f8ed920bdf | |
| chainwalkchainwalk/3 | llama.cppextraction | 50.0% | Coverage 2/2 | g f4bfcf6f040b0978d 129a9e74408c8919c 6e9183b6e3dfb175 | |
| contractcontract/2 | llama.cppextraction | 25.0% | Coverage 48/48 | g cc2d23c5bae45474d 1fe53d30cb5480f6c 5b96e006022e6218 | |
| screeningscreening/1 | llama.cppextraction | 96.7% | Coverage 60/60 | g 0661b49a10be2d2ad 64c97ca2269c29afc 2f28c0b25cd855b3 | |
| toolstools/1 | llama.cppextraction | 80.0% | Coverage 15/15 | g 2763675a5c3da564d 33a44812f0ad4c39c eb102ab31b1fed3f | |
| toolscaletoolscale/1 | llama.cppextraction | 95.0% | Coverage 40/40 | g a2bc3751abfa28e0d b24abb108c409178c 3cdd864ed9d75224 | |
| toolcalltoolcall/1 | llama.cppextraction | 0.0% | Coverage 60/60 | g 804e8fc9457649d8d 10af88c170513155c 45961d58b222b2b6 | |
| nullschemanullschema/1 | llama.cppextraction | 48.6% | Coverage 72/72 | g d7448e0f5fc3f9afd 3b91280bb40372efc 8dc1ad6df730d117 | |
| worldviewworldview/2 | llama.cppextraction | 20.0% | Coverage 40/40 | g 9acfba1b704feb28d e97835de9e6fc227c 8917e280c32ae81f | |
| synthesissynthesis/2 | llama.cppextraction | 28.3% | Coverage 60/60 | g 4156dd3c13361e8bd 267d1954e225c5b6c deca2750b95cfb02 | |
| epistemicsepistemics/4 | llama.cppextraction | 72.9% | Coverage 48/48 | g acf84972a0ad5f38d 4bce453fd376553cc 46daddd78b43dd51 | |
| enrichmentenrichment/3 | llama.cppextraction | 77.8% | Coverage 54/54 | g b51222c2ca38cfdfd 75361864baa727abc a398dac440b5228c | |
| groundinggrounding/1 | llama.cppextraction | 72.8% | Coverage 125/125 | g 30b933dcc6a7e2d4d dcb396faf8b83b0ac aaa3ffccbf9aaeb4 | |
| needleneedle/1 | llama.cppextraction | 63.6% | Coverage 110/110 | g c38dec33aa1c86c2d 5c5199b9eee8a595c 2dc8e6d07655daca | |
| tagger_nullabletagger-nullable/1 | llama.cppextraction | 88.8% | Coverage 240/240 | g 04e875431e26c4b5d db3addf23a2bc4cec 81db34b0e26e0e30 | |
| taggertagger/2 | llama.cppextraction | 87.5% | Coverage 240/240 | g 63e04d139a0e5ed3d 7ef587bf33e5788ac 1ce3949b6664155c |