Qwen3.5-9B · file← Qwen3.5-9B

Q4_K_M

Unsloth's Q4_K_M, the 4-bit half. It matched the Q8_0, then became the starting point for the format experiments. Unsloth GGUF.

Size: 5.7 GB
Status: Deleted 12 Sep 2026
Scored runs: 1
Smallest Mac: 16 GB estimate, 8K context

Results1 scored · 95% intervals

Suite	Score	95% interval	Passed	Runtime · profile	Recorded
nullschemaqwen35-9b-gguf-q4km	23.6%	15.3–34.6	Passed 17/72	llama.cppextraction	2026-09-11

Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.

The recordHow it was made · how it was served · what went wrong · what happened to it

Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.

File	`qwen3.5-9b/Qwen3.5-9B-Q4_K_M.gguf`
Format	Q4_K_M, 5,680,522,464 B
Where from	downloaded 2026-08-23, same repo
Status	deleted 2026-09-12 07:45Z
Hugging Face	same

Where it came from

Repo / file: unsloth/Qwen3.5-9B-GGUF / Qwen3.5-9B-Q4_K_M.gguf, resolve/main, fetched by the same get.sh as the Q8_0. Revision not on record.
Size: 5,680,522,464 B (HF listing 5.68 GB). llama-bench tensor model_size 5,669,554,176 B. sha256 not on record.
Who asked / why: the "q4" half of "q8 and then q4". Q4_K_M is the default quant, so it stood for "the usual 4-bit".
The same file was later the source of the made-here Q4_0 (Qwen3.5-9B Q4_0 (requantised here from the local Q4_K_M)).

How it was served

label	dates (UTC)	binary and version	ctx / -np / KV / -fa / `--jinja` / `--reasoning-format`	harness profile	why this label exists
`qwen3.5-9b-q4__study-nothink` (eval v1, not in the run log)	2026-08-23 19:14-20:32	`llama.cpp` llama-server, b10369 by reconstruction	`-ngl 99 -c 73728 -np 6 --jinja -cb`	`study-nothink`; configuration_id `32ddcb98599486fb`	the Q4 arm of the Q8-vs-Q4 question; 77.8 min
(llama-bench, no label)	2026-08-23 13:07	b10369	`-p 512 -n 128 -r 5`	n/a	fleet baseline: `pp512` 353.59, `tg128` 31.74
(llama-bench, no label)	2026-09-11 13:20	`llama.cpp/build/bin/llama-bench`, b10893	`-p 0 -n 64 -ngl 99 -r 2`	n/a	baseline for the Q4_0 sweep: 31.24 tok/s
(llama-bench, no label)	2026-09-11 14:44	b10893	`-p 0 -n 128 -ngl 99 -r 3`	n/a	`three_way2.sh`: 21.60 tok/s
`qwen35-9b-gguf-q4km` (run log)	2026-09-11 15:28 (BST 16:28:57)	llama-server b10893 (`c32d1dabe`) from /props	16384 / 1 / q8_0 / on / not on the command line / not set; model id was the file path	`extraction`, `--no-resume`, nullschema only	baseline arm of the five-format nullschema experiment (Q4_K_M, Q4_0, three MLX builds), a one-off script
(llama-bench, llama-cli, no label)	2026-09-11 21:03 and 21:49	b10893	llama-cli `-c 4096 -ngl 99 -n 192 --temp 0`; llama-bench `-p 0 -n 64 -r 2`	n/a	reference arm of the alignment sweep: 32.8 tok/s (cli), 31.45 +/- 0.03 (bench)

Seats held: none.

Decisions

Downloaded as the Q4 arm.
Kept after the study because "Q4 costs nothing in quality" and saves 3.85 GB.
Used as the requantisation source for the Q4_0 on 2026-09-11.
Deleted 2026-09-12 07:45:56Z with the rest.

What went wrong

The nullschema score is mostly a phrasing artefact. The GGUF servers declined by writing "not mentioned", the grader's fabricated bucket takes any present non-empty value; the MLX builds of the same weights wrote empty strings and scored 59.7% (Runtime changes HOW a model phrases a refusal, and nullschema scores that as fabrication). The assistant first read the 23.6% as model weakness ("a weak abstainer") and then retracted it.
The 21.60 tok/s figure looks contended and is in a published finding. Three other measurements of the same file on the same build are 31.2-32.8 (table above); the 21.60 was taken while qwen27b_q40.sh (a harness run against a 27B server) was still active. Finding llama.cpp, mlx_lm and rapid-mlx decode at the same speed; only the format differs quotes 21.60 as the "outlier at half speed"; the clean ratio is about 1.4x. (reconstructed from the process list.)
Filename collision in the first Q4_0 sweep briefly made the 27B look 4.5x faster; it measured this 9B Q4_0 twice. Not a defect of this file.
Missing flag (standing check): the Sep 11 llama-server line had no --jinja and no --reasoning-format; /props reports cache_type_k and cache_type_v as null in the manifest although the command line sets q8_0 (benchmarks/results/nullschema__qwen35-9b-gguf-q4km__extraction.manifest.json, read 2026-10-23). Whether this build applies the chat template without --jinja is not on record.
Partial runs: none (72/72). Stale build stamp: run-log b10893 c32d1dabe matches /props and llama-bench.

What happened to it

Downloaded 2026-08-23 -> eval v1 and llama-bench the same day -> source of the Q4_0 and the format experiments 2026-09-11 -> deleted 2026-09-12 07:45:56Z. No RESTORE-deleted-*.sh entry exists for it. Re-downloadable from unsloth/Qwen3.5-9B-GGUF.