Qwen3.5-9B · file← Qwen3.5-9B
Q8_0
Unsloth's Q8_0, the more precise half of the Q8-against-Q4 test. More of the scored runs are on this file than any other. Unsloth GGUF.
- Size
- 9.5 GB
- Status
- Deleted 12 Sep 2026
- Scored runs
- 4
- Smallest Mac
- 18 GB estimate, 8K context
Results4 scored · 95% intervals
| Suite | Score | 95% interval | Passed | Runtime · profile | Recorded |
|---|---|---|---|---|---|
| groundingqwen3.5-9b-q8 | 72.0% | 63.6–79.1 | Passed 90/125 | llama.cppextraction | 2026-08-24 |
| groundingqwen3.5-9b-q8-b10618 | 72.0% | 63.6–79.1 | Passed 90/125 | llama.cppextraction | 2026-08-25 |
| nullschemaqwen3.5-9b-q8-b10618 | 30.6% | 21.1–42.0 | Passed 22/72 | llama.cppextraction | 2026-08-25 |
| screeningqwen3.5-9b-q8-b10618 | 96.7% | 88.6–99.1 | Passed 58/60 | llama.cppextraction | 2026-08-25 |
Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.
The recordHow it was made · how it was served · what went wrong · what happened to it
Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.
| File | qwen3.5-9b/Qwen3.5-9B-Q8_0.gguf |
| Format | Q8_0, 9,527,502,048 B |
| Where from | downloaded 2026-08-23 from unsloth/Qwen3.5-9B-GGUF |
| Status | deleted 2026-09-12 07:45Z |
| Hugging Face | https://huggingface.co/unsloth/Qwen3.5-9B-GGUF |
Where it came from
- Repo / file:
unsloth/Qwen3.5-9B-GGUF/Qwen3.5-9B-Q8_0.gguf, fromresolve/main; revision not on record. The repo was last updated 2026-03-02 on HF's listing of 2026-08-24. - Command:
cd models/qwen3.5-9b; B=https://huggingface.co/unsloth/Qwen3.5-9B-GGUF/resolve/main; for f in Qwen3.5-9B-Q8_0.gguf Qwen3.5-9B-Q4_K_M.gguf mmproj-F16.gguf; do $PG "$B/$f" "$f" 6; done(pget.sh, 6 byte ranges). It reportedsize 9086 MiB in 6 parts. - Landed: between 10:20Z and 10:34Z; by 10:34:32Z the directory held 15 GB (all three files).
- Size: 9,527,502,048 B (HF listing 9.53 GB). llama-bench's tensor-only
model_sizeis 9,516,533,760 B. sha256 not on record. - Who asked / why: Alan, "put it through its paces at q8 and then q4". Q8_0 was named first because the question was Q8 against Q4.
How it was served
| label | dates (UTC) | binary and version | ctx / -np / KV / -fa / --jinja / --reasoning-format | harness profile | why this label exists |
|---|---|---|---|---|---|
qwen3.5-9b-q8__study-default (eval v1, not in the run log) | 2026-08-23 ~10:56-11:27 | llama.cpp llama-server, build not printed in the log; b10369 6e62ba538 by reconstruction (the rebuild to b10618 happened 2026-08-25) | -ngl 99 -c 73728 -np 6 --jinja -cb: ctx 73,728, 6 slots, KV default, -fa not set, --jinja on, no reasoning-format | study-default: thinking on, max_tokens=10000; configuration_id 3bd2d0f8420be3ab | the original brief: 624-task study, Q8 first. Paused at 18 rows |
qwen3.5-9b-q8__study-nothink (eval v1, not in the run log) | 2026-08-23 18:13-19:14 | as above | as above | study-nothink; configuration_id 875741d6d40e5d39 | the thinking-off arm, run first by Alan's "one at a time"; finished in 61.0 min |
(Toolery run 2026-08-24T11-29_qwen3.5-9b-q8, not in the run log) | 2026-08-24 11:29-~12:25 | b10369 | -ngl 99 -c 32768 -np 4 -cb --jinja | Toolery --adapter raw --tier all --trials 1 --concurrency 4 | tool-use ranking of the small models |
qwen3.5-9b-q8 (run log) | 2026-08-24 13:03 (BST 14:03:47) | llama.cpp b10369 (6e62ba538), run-log source "evidence" | -ngl 99 -c 32768 -np 4 -cb --jinja; -fa and KV type not set | extraction, --no-resume | the first grounding/1 run on the rebuilt immutable harness, queued behind the Toolery sweep |
qwen3.5-9b-q8-b10618 (run log) | 2026-08-25 10:44-10:47 (BST 11:44:57-11:47:31) | llama.cpp b10618 (eb25b7263) | -ngl 99 -c 32768 -np 4 -cb --jinja | extraction, --no-resume; grounding, screening, nullschema | fleet re-run after the llama.cpp rebuild, so every model had rows on one build (Alan: "keep benchmarking, we need to be keeping a run log too"). The same re-run also ran Toolery 2026-08-25T10-15_qwen3.5-9b-q8-b10618 |
| (llama-bench, no label) | 2026-08-23 13:08 | llama-bench b10369 | -p 512 -n 128 -r 5, f16 KV, -fa auto | n/a | the fleet baseline |
Seats held: none. It never appeared in fleet-models.ini, fleetview or launchd (grep -i qwen3.5-9b over both ini copies, 2026-10-23: only a comment about a different fine-tune).
Decisions
- Downloaded to answer "Q8 and then Q4".
- Kept as the better default of the two quants by the 2026-08-23 reading: "Q8 is the better default" if memory allows. Never seated, so the point was moot.
- Deleted 2026-09-12 07:45:56Z on Alan's "Kill qwen 3.5 9b and everything to do with it", with its siblings.
What went wrong
- Thinking-on run abandoned at 18/624 (Alan's reorder): the chain would have taken ~14 hours. Four of the 18 rows were truncated at the 10,000-token safety ceiling (
summary.json), i.e. unmeasured, not failures. - First chain failed instantly:
runner.pygot the profile and model names concatenated by a zsh splitting mistake (unknown profile), then fixed. Minutes only. - Toolery timeouts counted as failures. Fourteen of 143 scenarios hit the 80-second wall; the assistant's correction: "A large part of what I showed you as capability was speed on this box.".
- Eval-v1 totals disagree between the working session and
summary.json: ifeval 169/199 (Q8) and 165/198 (Q4) on 2026-08-23 against 179 and 175 in the 2026-10-09 file. The other three benches match exactly. Cause not on record (the grader or answer key may have changed during the repo migration). - An older lab note contradicted. The lab's day report for 2026-08-23 says quantisation buys nothing on this model and ties the plateau to the linear-attention layers. A Q4_0 of the same weights decodes at 44.6 t/s (llama-bench).
- Standing checks. Partial runs: none (grounding 125/125, screening 60/60, nullschema 72/72;
wc -l benchmarks/results/*qwen3.5-9b-q8*, 2026-10-23). Stale build stamps: run-log b10369 forqwen3.5-9b-q8is stamped from a script written at 07:39Z, before the run at 13:03Z; no rebuild intervened (the next was 2026-08-25 07:32Z), so it holds. Missing flags:--jinjapresent;-faand KV type left at defaults;--reasoning-formatnot set. Theextractionprofile's thinking setting was not re-read here; the run log records profile names only.
What happened to it
Downloaded 2026-08-23 -> eval v1, Toolery, grounding, b10618 re-run 2026-08-23..25 -> never seated -> deleted 2026-09-12 07:45:56Z. No RESTORE-deleted-*.sh entry exists for it. Re-downloadable from unsloth/Qwen3.5-9B-GGUF (Qwen3.5-9B-Q8_0.gguf; current revision may differ).