Qwen3.5-9B · file← Qwen3.5-9B
Q4_0
A Q4_0 we made from the Q4_K_M. About 1.4 times faster at writing than the Q4_K_M. Requantised here from the local Q4_K_M.
- Size
- 5.3 GB
- Status
- Deleted 12 Sep 2026
- Scored runs
- 1
- Smallest Mac
- 16 GB estimate, 8K context
| Suite | Score | 95% interval | Passed | Runtime · profile | Recorded |
|---|---|---|---|---|---|
| nullschemaqwen35-9b-gguf-q40 | 6.9% | 3.0–15.2 | Passed 5/72 | llama.cppextraction | 2026-09-11 |
Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.
Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.
| File | _q40test/Qwen3.5-9B-Q4_K_M__Q4_0.gguf, renamed qwen3.5-9b/Qwen3.5-9B-Q4_0.gguf |
| Format | Q4_0, about 5.31 GB (4.94 GiB) |
| Where from | made here 2026-09-11 from the local Q4_K_M |
| Status | deleted 2026-09-12 07:45Z |
How it was made
- Input:
models/qwen3.5-9b/Qwen3.5-9B-Q4_K_M.gguf(unsloth, 5,680,522,464 B; Qwen3.5-9B Q4_K_M (unsloth GGUF)). The file is therefore rounded twice (BF16 -> unsloth Q4_K_M -> Q4_0). The assistant noted on the same afternoon that requantising an already-quantised file "carries two rounding passes". - Tool:
llama.cpp/build/bin/llama-quantize --allow-requantize <in> <out> Q4_0, build b10893 (c32d1dabe) by the run log. The exact moment of creation is not pinned: the first sweep produced a Q4_0 of the 9B under the collided name_q40test/Qwen3-Q4_0.gguf(4.94 GiB, benched at 43.84 and 44.16 tok/s), which the assistant then deleted ("removed the collided file").three_way2.sh(queued 14:19Z) re-created it asQwen3.5-9B-Q4_K_M__Q4_0.ggufthrough its fallbackllama-quantize --allow-requantize "$GQ4KM" "$GQ40" Q4_0("Q4_0 missing - requantising"); a one-off script (written 13:25Z) has the same fallback but is recorded as "(not started)". So the surviving file dates from about 14:20-14:44Z. (reconstructed) - Renamed 2026-09-11 15:51Z to
qwen3.5-9b/Qwen3.5-9B-Q4_0.gguf, after the filename__Q4_0was judged to hide provenance. - Who asked / why: Alan, "we have a lever, chase it, measure it, prove it", after the Q4_0 speed lever showed on granite-guardian. The 9B was one of the five sweep models.
How it was served
| label | dates (UTC) | binary and version | ctx / -np / KV / -fa / --jinja / --reasoning-format | harness profile | why this label exists |
|---|---|---|---|---|---|
| (llama-bench, no label) | 2026-09-11 13:20 | b10893 | -p 0 -n 64 -ngl 99 -r 2 | n/a | the collided first copy: 43.84 tok/s against 31.24 for the Q4_K_M |
| (llama-bench, no label) | 2026-09-11 14:44 | b10893 | -p 0 -n 128 -ngl 99 -r 3 | n/a | three_way2.sh: 44.11 tok/s |
qwen35-9b-gguf-q40 (run log) | 2026-09-11 15:29 (BST 16:29:50) | llama-server b10893 (c32d1dabe) | 16384 / 1 / q8_0 / on / not on the command line / not set; model id the _q40test path | extraction, --no-resume, nullschema only | Q4_0 arm of the five-format nullschema experiment |
| (llama-bench, no label) | 2026-09-11 21:49 | b10893 | -p 0 -n 64 -r 2 | n/a | the independent check using the quantlab copy: 44.59 +/- 0.01 |
Seats held: none.
Decisions
- Made to price the Q4_0 speed lever on a hybrid-attention dense model; kept as the Q4_0 arm of the format experiment.
- Renamed on 2026-09-11 and listed among the "Q4_0 builds" to keep.
- Deleted 2026-09-12 07:45:56Z. Not published to HF, no reason on record.
What went wrong
- Filename collision in the first sweep:
${N:r:r}collapsedQwen3.8-27B-UD-Q4_K_M.ggufandQwen3.5-9B-Q4_K_M.ggufto one name; the 27B result was the 9B measured again. This file is not defective; the collided first copy was deleted. - Phrasing artefact: the 6.9% nullschema score is a convention effect, not collapse.
- Wrong initial reading: "That inverts the finding" was withdrawn within minutes.
- Missing flag (standing check): no
--jinja, no--reasoning-format, cache types null in the manifest although the command line sets q8_0. - Partial runs: none (72/72). Build stamp: b10893 consistent.
What happened to it
Made 2026-09-11 (about 14:20-14:44Z) -> benched and nullschema-scored same day -> renamed 15:51Z -> deleted 2026-09-12 07:45:56Z. No RESTORE-deleted-*.sh entry exists for it. Re-creatable only as BF16 -> unsloth Q4_K_M -> Q4_0 with the commands above; a single-pass BF16 -> Q4_0 would be a different artefact. The quantlab Q4_0 (Qwen3.5-9B Q4_0 (requantised here from the downloaded Q8_0, "quantlab")) is a different file (from the Q8_0).