Q4_0

The one we kept. Made here from the Q4_K_M, and about 1.5 times faster. Requantised here from the UD-Q4_K_M.

Size: 15.7 GB
Status: On disk
Scored runs: 5
Smallest Mac: 32 GB estimate, 8K context

Results5 scored · 95% intervals

Suite	Score	95% interval	Passed	Runtime · profile	Recorded
nullschemaqwen3.8-27b-q4_0	69.4%	58.0–78.9	Passed 50/72	llama.cppextraction	2026-09-11
synthesisqwen3.8-27b-q4_0	83.3%	72.0–90.7	Passed 50/60	llama.cppextraction	2026-09-11
toolcallqwen3.8-27b-q4_0	100.0%	94.0–100.0	Passed 60/60	llama.cppextraction	2026-09-11
toolscaleqwen3.8-27b-q4_0	100.0%	91.2–100.0	Passed 40/40	llama.cppextraction	2026-09-11
worldviewqwen3.8-27b-q4_0	80.0%	65.2–89.5	Passed 32/40	llama.cppextraction	2026-09-11

Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.

The recordHow it was made · how it was served · what went wrong · what happened to it

Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.

File	`Qwen3.8-27B-Q4_0.gguf`
Format	Q4_0, 15,715,855,264 B
Where from	made here 2026-09-11 from the UD-Q4_K_M
Status	on disk; `qwen3.8` bench occupant of the swap seat

How it was made

Input: models/qwen3.8-27b-gguf/Qwen3.8-27B-UD-Q4_K_M.gguf (see Qwen3.8-27B UD-Q4_K_M (unsloth GGUF, MTP head embedded)), a mixed-precision Unsloth Dynamic file, not a uniform Q4_K_M. So this is a second rounding of an already mixed quant.
Command:

llama.cpp/build/bin/llama-quantize --allow-requantize <UD-Q4_K_M.gguf> models/_q40test/Qwen3.8-27B-UD-Q4_K_M__Q4_0.gguf Q4_0

Tool: llama.cpp llama-quantize, build 0.4.0-dev b10893 (c32d1dabe; the upgrade of 2026-09-10).
Date: 2026-09-11, 13:21Z (the file's mtime is 14:21 BST, preserved by the later mv).
Size: 15,715,855,264 B (stat). sha256 not on record.
Who asked / why: the Q4_0 sweep itself came out of the byte-alignment speed work of that day (Byte alignment, not bit count, governs decode speed).
Rename: _q40test/Qwen3.8-27B-UD-Q4_K_M__Q4_0.gguf became qwen3.8-27b-gguf/Qwen3.8-27B-Q4_0.gguf on 2026-09-11 15:51Z, the old name having claimed both formats in one filename.

How it was served

label	dates (UTC)	binary and version	ctx / -np / KV / -fa / --jinja / --reasoning-format	harness profile	why this label exists
`qwen3.8-27b-q4_0`	2026-09-11 13:52–14:16	`llama.cpp` b10893 (`c32d1dabe`)	`-c 32768 -np 1 -fa on -ctk q8_0 -ctv q8_0 -ngl 99`; no `--jinja` and no `--reasoning-format` on the line	extraction	the second model for the Q4_0 quality check after K2-36B: a dense, strong abstainer with a full baseline row, queued behind the K2 format ladder

Later, unlabelled servings: 2026-09-11 15:14Z with 32768 context and q8_0 KV for the speculative-drafter tests; 2026-09-28 with -c 65536 -np 1 -ctk q4_0 -ctv q4_0 -fa on --jinja --alias qwen38-27b as the llama.cpp arm of the Splash comparison and with -np 4 for the concurrency arm. Seats held: none as a standing seat; a registered qwen3.8 bench occupant of the swap seat in fleetview.py (read 2026-10-23).

Decisions

Made to price the Q4_0 speed lever on a dense model with a full baseline (script header: dense should get about 1.4 times, and the model is a strong abstainer).
First verdict (withdrawn): 'requantise the fleet to Q4_0 is not supported'. Second verdict: 'Q4_0 does not change abstention judgement at all', it changes output conventions.
Seat recommendation of 2026-09-11 15:50Z: qwen3.8-27b Q4_0 as 'writer', 15.2 tok/s, toolcall 100.0, worldview 80.0, nullschema 69.4, synthesis 83.3.
Kept under Alan's "keep any q4_0 and drafters". Not published as its own model; its rows are the qwen3.8-27b-q4_0 label under the qwen3.8-27b write-up.

What went wrong

Name collision in the first sweep. zsh ${N:r:r} made both the 27B and the 9B write Qwen3-Q4_0.gguf; the 27B step skipped quantising and benched the 9B, producing a 4.5 times 'speedup' that the llama-bench model-name column exposed. Fixed by full-basename names.
Grader convention misread as capability. The -7.0 nullschema change was a phrasing difference (words versus empty field), scored as fabrication by the grader. The harness gap (a verbal decline counts as fabricated) is recorded in the lab's notes, not fixed in place (the harness is locked).
Missing flags (standing check): the benchmark launch line has no --jinja and no --reasoning-format, unlike the -np1 family of labels. Whether the build's default made --jinja redundant is not on record.
Build stamp: run-log b10893 matches the evidence.

What happened to it

Made 2026-09-11 13:21Z → benched same afternoon → renamed 15:51Z → registered as the qwen3.8 bench entry of the swap seat → on disk at models/qwen3.8-27b-gguf/Qwen3.8-27B-Q4_0.gguf (read 2026-10-23). Re-creatable from Qwen3.8-27B UD-Q4_K_M (unsloth GGUF, MTP head embedded) plus the command above.