Spark-X2.5-4B · file← Spark-X2.5-4B

Q4_K_M

A fresh download of XHToken's 4-bit file on 24 October, for a fuller re-run. Where the runs overlap, it scored exactly the same.

Size: 2.6 GB
Status: Benched 24 Oct 2026
Scored runs: 15
Smallest Mac: 16 GB measured, 64K context

Results15 scored · 95% intervals

Suite	Score	95% interval	Passed	Runtime · profile	Recorded
chainwalkspark-x2.5-4b-q4km-np1-c64k-b11846	50.0%	9.5–90.5	Passed 1/2	llama.cppextraction	2026-10-24
contractspark-x2.5-4b-q4km-np1-c64k-b11846	25.0%	14.9–38.8	Passed 12/48	llama.cppextraction	2026-10-24
enrichmentspark-x2.5-4b-q4km-np1-c64k-b11846	77.8%	65.1–86.8	Passed 42/54	llama.cppextraction	2026-10-24
epistemicsspark-x2.5-4b-q4km-np1-c64k-b11846	72.9%	59.0–83.4	Passed 35/48	llama.cppextraction	2026-10-24
groundingspark-x2.5-4b-q4km-np1-c64k-b11846	72.8%	64.4–79.8	Passed 91/125	llama.cppextraction	2026-10-24
needlespark-x2.5-4b-q4km-np1-c64k-b11846	63.6%	54.3–72.0	Passed 70/110	llama.cppextraction	2026-10-24
nullschemaspark-x2.5-4b-q4km-np1-c64k-b11846	48.6%	37.4–59.9	Passed 35/72	llama.cppextraction	2026-10-24
screeningspark-x2.5-4b-q4km-np1-c64k-b11846	96.7%	88.6–99.1	Passed 58/60	llama.cppextraction	2026-10-24
synthesisspark-x2.5-4b-q4km-np1-c64k-b11846	28.3%	18.5–40.8	Passed 17/60	llama.cppextraction	2026-10-24
taggerspark-x2.5-4b-q4km-np1-c64k-b11846	87.5%	82.7–91.1	Passed 210/240	llama.cppextraction	2026-10-24
tagger_nullablespark-x2.5-4b-q4km-np1-c64k-b11846	88.8%	84.1–92.2	Passed 213/240	llama.cppextraction	2026-10-24
toolcallspark-x2.5-4b-q4km-np1-c64k-b11846	0.0%	-0.0–6.0	Passed 0/60	llama.cppextraction	2026-10-24
toolsspark-x2.5-4b-q4km-np1-c64k-b11846	80.0%	54.8–93.0	Passed 12/15	llama.cppextraction	2026-10-24
toolscalespark-x2.5-4b-q4km-np1-c64k-b11846	95.0%	83.5–98.6	Passed 38/40	llama.cppextraction	2026-10-24
worldviewspark-x2.5-4b-q4km-np1-c64k-b11846	20.0%	10.5–34.8	Passed 8/40	llama.cppextraction	2026-10-24

Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.

SetupHow it was run

Of record

The measurement of record for this model.

The run’s own note: Single slot at 64K (benchmark rule), beside the full fleet: the training comparison for Spark-X2.6-4B. Weights re-downloaded and pinned to a revision; the vendor Q4_K_M here is a plain Q4_K/Q6_K mix. On the five suites run on 2026-09-10, 270 of 272 answers are byte-identical to that run. shapegen and sitebuild excluded.

Server: 1 slot, 64K of context per request, KV cache f16 (default), flash attention on, 1 suite at a time
Runtime: llama.cpp b11846 (1a24df163), plain upstream
Speed: decode 58.4 tokens/s, prefill 453.2 tokens/s (433.2 on prompts over 16K), medians. Single stream, full fleet resident (contended).
Peak memory: 7 GB resident, with 64K of context allocated
Weights: 2.6 GB file · sha256 adfcfa19a4ed6a59…
Coherence check: passed
Left out by plan: shapegen, sitebuild
Machine: Mac Studio (M1 Max, 64 GB)
Released: 24 October 2026, bundle 2026-10-24-0b6bf0e3

Memory by context

The same file and server, started at each context size in turn: what it held in memory and how fast it wrote on a short prompt. It measures memory and serving, not how well the model uses a long context. Released 24 October 2026, bundle 2026-10-24-eaf7f863.

The ladder’s own note: Context ladder 8K to 256K (native window 1M; the ladder tops out at 262144), single slot, f16 KV, beside the full fleet. Decode on a trivial prompt per rung; measures allocation and serving, not long-context quality.

Context	Resident memory	Swap in use, whole Mac	Decode	Load
8K	2.93 GB	10.02 GB	59.7 tokens/s	3 s
16K	3.22 GB	10.02 GB	59.7 tokens/s	3 s
32K	3.78 GB	10.02 GB	59.6 tokens/s	3 s
64K	4.91 GB	10.02 GB	59.5 tokens/s	3 s
128K	7.16 GB	10.02 GB	59.7 tokens/s	3 s
256K	11.62 GB	10.02 GB	59.5 tokens/s	6 s

The recordHow it was made · how it was served · what went wrong · what happened to it

Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.

File	`Spark-X2.5-4B-Q4_K_M.gguf` (re-download)
Format	Q4_K_M GGUF, 2,600,224,352 B (HF listing 2026-10-24)
Where from	downloaded 2026-10-24, pinned to revision `9826e0be84e6e6e8b9668abc91421109a1df1e2d`
Status	benched 2026-10-24
Hugging Face	https://huggingface.co/XHToken/Spark-X2.5-4B-GGUF

How it was made

Downloaded, not made here.

field	value
repo	`XHToken/Spark-X2.5-4B-GGUF` (repo last modified 2026-09-20T08:08:37Z)
file	`Spark-X2.5-4B-Q4_K_M.gguf`
revision	`9826e0be84e6e6e8b9668abc91421109a1df1e2d`
size	2,600,224,352 bytes (`ls -l`; same as the HF listing, and as the deleted 2026-09-10 file)
sha256	`adfcfa19a4ed6a5985da8bf565fe15f8e1a7e131d79bae2d19d48d1c40109428` (`shasum -a 256`; equals the HF LFS oid)
downloaded	2026-10-24 ~15:30Z, `huggingface_hub` 1.33.0 with `HF_XET_HIGH_PERFORMANCE=1`
local path	`models/spark-x2.5-4b/Spark-X2.5-4B-Q4_K_M.gguf`
asked by	Alan, 2026-10-24: "yes re-run"

Identity with the 2026-09-10 file:

Bit identity is not provable. The byte size is the same, but that file's sha256 was never recorded, and the repo was modified after the first download.
Behaviourally it is the same file. On the five suites both runs share, 270 of 272 answers are byte-identical, and all 272 pass/fail outcomes match. That holds across a different llama.cpp build, context and KV type.

Header (gguf-py, llama.cpp-b11846):

architecture spark2_5, 36 blocks, 4 KV heads, context_length 1,048,576, general.file_type 15 (Q4_K_M).
Tensor mix:

type	tensors	bytes
Q4_K	180	1.70 GB, incl. every `attn_output`
Q6_K	37	0.89 GB, incl. `token_embd.weight`
F32	73	~0 (norms)

There's no separate output.weight tensor; the output head appears tied to the embeddings.
Against X2.6's vendor "Q4_K_M": that file has BF16 token_embd, Q8_0 attn_output and a separate Q8_0 output.weight, and is 3,543,441,312 B. So the two vendor files at the same nominal format differ in precision, and possibly in whether the output head is tied.

How it was served

Machine: a Mac Studio (Mac13,1): Apple M1 Max, 64 GB.

label	dates (UTC)	binary and version	ctx / -np / KV / -fa / --jinja / --reasoning-format	profile	why this label exists
`spark-x2.5-4b-q4km-np1-c64k-b11846`	2026-10-24 15:41Z–17:50Z	`llama.cpp-b11846/build/bin/llama-server`, build 11846 (`1a24df163`)	`-c 65536 -np 1` / KV default f16 / `-fa on` / `--jinja` / default / `-ngl 999 --cache-ram 0`	extraction	the training-comparison re-run (the family's decision log, 2026-10-24); 16 suites incl. toolcall2, `PAR=1`, `VACATE=0`; coherence COHERENT
`spark-x2.5-4b-q4km-np1-b11846`	2026-10-24 17:50Z–17:51Z	same binary	`-np 1 -fa on`, f16 KV, 8K→256K	—	context ladder

Decisions

2026-10-24: re-downloaded and pinned for the re-run (family decision log).
2026-10-24: re-run complete; expectations scored in the family doc.

What went wrong

Mis-stamped run log (known in advance): llama_build reads the live binary and server_flags is empty. The truth is build 11846 with the Served-as flags. The bundle carries the true runtime_version.
The waiting task hit its 2-hour limit at the end of the queue; the queue itself finished (QUEUE_DONE 17:51:41Z). Nothing was lost.

What happened to it

Downloaded 2026-10-24 → benched 2026-10-24. Re-downloadable from the repo and revision above. Not seated.