Qwen3-8B · file← Qwen3-8B

Q4_0

The one we kept, made here from the Q4_K_M. In our model router since 11 September; one test since. Requantised here from the local Q4_K_M.

Size: 4.8 GB
Status: On disk
Scored runs: 1
Smallest Mac: 16 GB estimate, 8K context

Results1 scored · 95% intervals

Suite	Score	95% interval	Passed	Runtime · profile	Recorded
nullschemaqwen3-8b-b11552	47.2%	36.1–58.6	Passed 34/72	llama.cppextraction	2026-10-11

Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.

The recordHow it was made · how it was served · what went wrong · what happened to it

Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.

File	`models/qwen3-8b-gguf/Qwen3-8B-Q4_0.gguf`
Format	GGUF Q4_0, 4,774,750,016 B
Where from	made here 2026-09-11 13:19Z from the Q4_K_M
Status	on disk; router `[qwen3-8b]` preset

How it was made

Input: models/qwen3-8b-gguf/Qwen3-8B-Q4_K_M.gguf (the Unsloth Q4_K_M, see Qwen3-8B Q4_K_M (unsloth GGUF)). The output has therefore been rounded twice (BF16 -> Q4_K_M -> Q4_0); the header's Unsloth imatrix keys are inherited metadata, not an imatrix applied to this step.
Command:

Q=llama.cpp/build/bin/llama-quantize
DST=$OUT/${N:r:r}-Q4_0.gguf # N=Qwen3-8B-Q4_K_M.gguf -> _q40test/Qwen3-8B-Q4_K_M-Q4_0.gguf
$Q --allow-requantize "$SRC" "$DST" Q4_0

with OUT=models/_q40test. The binary was the production llama.cpp/build/bin (b10893, c32d1dabe, upgraded 2026-09-10 12:17Z). The quantize log line for this file is not on record (the monitor filtered it out).

Date: file mtime 2026-09-11 14:19:33 BST = 13:19:33Z, nine seconds after the sweep launched.
Size / identity: 4,774,750,016 B; sha256 203bbddaca2b41dfc23cc1eb84ee0ca0b55904ba3f2861cbe4a2b6bf2bc59923 (stat and shasum -a 256, 2026-10-23). general.file_type = 2, general.quantized_by = Unsloth.
Who asked / why: Alan, 13:18Z: "we have a lever, chase it, measure it, prove it", after granite-guardian showed Q4_0 at 52.22 tok/s against 36.84 for Q4_K_M on identical weights. This model was one of five in the first sweep (K2-3.7B, Qwen3-8B, Qwen3.5-9B, Qwen3.8-27B, K2-36B).
Renamed 2026-09-11 15:51Z: mv _q40test/Qwen3-8B-Q4_K_M-Q4_0.gguf qwen3-8b-gguf/Qwen3-8B-Q4_0.gguf in the cleanup that rehomed all six Q4_0 builds.
A subagent report of 2026-10-07 classed this file "Probably made locally, but not certain"; the sweep command above settles it.

How it was served

label	dates (UTC)	binary and version	ctx / -np / KV / -fa / `--jinja` / `--reasoning-format`	harness profile	why this label exists
`qwen3-8b-b11552`	2026-10-11 ~11:5x (run-log local 12:02:38)	`llama.cpp-b11552/build/bin/llama-server`, `0.6.0-dev (build 11552, commit 23b0202a1)`; run log `b11552 (23b0202a1)` from /props	8192 / not passed (server default) / f16 (default) / not passed (build default) / yes / not set; `--context-shift --n-gpu-layers 99 --alias qwen3-8b`	`extraction`	quality gate for the llama.cpp b11552 upgrade: nullschema on the router's qwen3-8b with "seat-identical flags". Label carries the build per rule 6

Gate command:

$BIN --model models/qwen3-8b-gguf/Qwen3-8B-Q4_0.gguf --jinja --context-shift --ctx-size 8192 --n-gpu-layers 99 \
     --alias qwen3-8b
python -m harness.runner --suite nullschema --model <server> --model-id qwen3-8b \
     --label qwen3-8b-b11552 --profile extraction

The flags are the router child's own: --context-shift --jinja --ctx-size 8192 --model... --n-gpu-layers 99, from [*] defaults (n-gpu-layers = 99, c = 8192, jinja = true) plus the preset in fleet-models.ini.

Seat held (outside the run log): router [qwen3-8b] preset, on demand (--models-max 1), 2026-09-11 22:28Z -> present. Loaded child seen as pid 81636, 5.7 GB RSS. Not a launchd seat.

Decisions

Made to price the Q4_0 speed lever across architectures.
Seated (router preset) on 2026-09-11 22:28Z: "Only Qwen3-8B has a Q4_0 already on disk, so that's the one seat I can switch without a download", config backed up. It is the router's only Q4_0 preset among the K-quant seats at that time. No quality gate preceded the switch; the inventory tool thereafter labelled it "SERVING 4.8 qwen3-8b-gguf Q4_0 active fleet seat".
Kept after the 2026-10-11 sweeps as a live router preset (a lab note: MLX 8-bit "superseded by the live Q4_0 GGUF preset").
Not published.

What went wrong

Confounded gate comparison. The b11552 gate compares 47.2% (this file, no -fa, f16 KV, 8192 ctx) with September's 34.7% (a different file, the Q4_K_M, with q8_0 KV and -fa on). The assistant called it "higher, probably also a configuration difference rather than the build" and listed it as "baseline 34.7%" in its summary table; the difference is not a build effect and not shown to be a quant effect.
Idle backend. The router served zero requests between 2 and 9 October; the qwen3-8b load seen on 2026-10-07 09:39Z was "a probe, not work".
Stale comment. fleet-models.ini still says "41.2 t/s at Q4_K_M, loads in ~1.5s" above the Q4_0 preset (line 86).
Provenance doubt that no longer holds. Before the sweep command was recovered, the file's origin was only "probably made locally".
Standing checks: no partial run; run-log build b11552 (23b0202a1) matches the gate script's binary; --jinja present; reasoning format unset, consistent with extraction.

What happened to it

Made 2026-09-11 13:19Z (_q40test/) -> renamed into qwen3-8b-gguf/ 2026-09-11 -> router preset 2026-09-11 22:28Z -> b11552 gate 2026-10-11 -> on disk (models/qwen3-8b-gguf/Qwen3-8B-Q4_0.gguf, 4,774,750,016 B, checked 2026-10-23). Not on HF. Re-creatable only from the Q4_K_M by the command above (--allow-requantize); the Q4_K_M is gone, so a rebuild needs a fresh Unsloth Q4_K_M download first, and the result would be a different artefact if the upstream file has changed.