Qwen3.5-9B

• 2026-08-23 10:16Z — pull the 9B, Q8 then Q4. Alan: "can you pull the 3.5 9b and put it through its paces at q8 and then q4?". Command: get.sh fetched Qwen3.5-9B-Q8_0.gguf, Qwen3.5-9B-Q4_K_M.gguf and mmproj-F16.gguf from unsloth/Qwen3.5-9B-GGUF with 6-range pget.sh. Q4_K_M rather than a UD quant or Q4_0 because the brief said "q4"; why unsloth rather than lmstudio-community or bartowski is not on record.
• 2026-08-23 10:22Z — add the MTP pair. Alan: "Yes, both quants with MTP". Reasoning given: a dense 9B reads ~6.8 GB per token, the case where speculation should finally pay. (The assistant's question; reasoning contemporaneous.) The MTP A/B itself was never run (no spec-type draft-mtp command on this model is on record).
• 2026-08-23 10:56Z — start with thinking on. The chain launched study-default (thinking on) for Q8 first, then Q4, then both thinking-off. Projected 14 hours.
• 2026-08-23 11:26Z — Alan reorders. "I don't think we should do it all in one run, I'd rather do it one at a time, I'm REALLY curious about the small model tests". The chain was killed at 18 of 624 rows and the thinking-off pair went later in a queue behind the lever sweep and the 12B needles.
• 2026-08-23 12:36Z — pre-registered expectation. If the 9B's Q4 and Q8 decode at the same speed it is "not bandwidth-bound at all", which would invert the speculation rule and make the MTP pair "the cleanest speculation A/B available anywhere". Outcome: the speed equality was real but the explanation was later overturned by the Q4_0 result; the A/B was not run.
• 2026-08-23 12:43Z — fleet llama-bench. Alan: "yeah llama bench across all models would be good, how long to run it though? Be a good baseline before messing around with my usefulness experiments". The 9B's four builds were four of 22 files benched (12:59-13:14Z).
• 2026-08-23 20:34Z — answer to the original question. Q4 costs nothing in quality or decode speed and saves 3.85 GB; "Q8 is the better default" if memory allows.
• 2026-08-24 — Toolery and grounding. Alan "go with recommendation" (drop the remaining Toolery models to one trial); the 9B ran 11:29Z-~12:25Z (Toolery) and 13:03Z (grounding). The assistant then corrected its own leaderboard for timeouts: "A large part of what I showed you as capability was speed on this box.".
• 2026-08-25 10:15Z — re-run on b10618. After the llama.cpp rebuild the whole fleet was re-run on one build; Alan, 08:05Z: "I’m out an about so keep benchmarking, we need to be keeping a run log too". The 9B's grounding was identical (72.0%).
• 2026-09-07 17:36Z — MTP directory deleted. The assistant listed the plain and MTP directories as "near-identical, both have Q8 and Q4" under duplicate quants; Alan: "Go for it and what’s the blocker on Xcode? I can be installed from cli". The delete script filed it under "superseded / measured-and-rejected", although no MTP measurement existed.
• 2026-09-11 13:18Z — the 9B becomes the format-experiment subject. "we have a lever, chase it, measure it, prove it". Q4_0 sweep: 31.24 to 43.84 tok/s, 1.40x. A filename-mangling bug first measured the 9B twice and produced a phantom 4.5x on the 27B.
• 2026-09-11 13:25Z — pre-registered (MLX matrix). The script header: "Prediction on record: GGUF Q4_0 beats MLX 4-bit.". Outcome: they tie (44.11 against 45.30 tok/s). The script was written but never started; the same question was answered by three_way2.sh and mlx_format_test.sh.
• 2026-09-11 15:28Z — pre-registered (format experiment). "The prediction, stated before the numbers": mxfp4 strongest, affine-g64 weakest. Outcome: mxfp4 worst, affine-g64 best.
• 2026-09-11 15:30Z — walk-back and walk-back of the walk-back. The assistant first said the 9B result "inverts the finding", then that the 9B is below the floor where the suite discriminates, then (15:40Z) that the whole comparison is a decline-phrasing confound.
• 2026-09-11 15:42Z — Alan. "sure and then I'd like to take a look at rapid, then draw conclusions on which models and runtimes for them I should plug into hermes". Context: the assistant had just proposed a post-hoc decline-form analyser; the line is about the runtime/format work this model was the subject of, not about seating it.
• 2026-09-11 15:51Z — rename. mv _q40test/Qwen3.5-9B-Q4_K_M__Q4_0.gguf qwen3.5-9b/Qwen3.5-9B-Q4_0.gguf.
• 2026-09-11 21:01Z — quantlab. Own quants at four widths to test the alignment law; a ternarisation pipeline built and measured. Alan's earlier prompt was a ternary question and, at 20:58Z, "No take the 27b-mtp for sure, I meant making our own": he wanted to make ternary models himself, not only download them.
• 2026-09-11 22:28Z — F16 intermediates removed. Authorisation (reconstructed): the assistant had offered it in a three-item list and Alan's next message at 22:27:58Z was "Tidy loose ends".
• 2026-09-12 07:44Z — kill it. Alan: "Kill qwen 3.5 9b and everything to do with it". Executed 07:45:56Z: the model directory, the five _quantlab outputs, the three _mlxfmt builds and the 18 GB HF cache, 87.9 GB in all. The assistant's note: "The numbers are recorded; the artefacts aren't.".

• Q8_0 against Q4_K_M, thinking off, on the 624-task study: no measurable difference. The assistant's reading on 2026-08-23: "no measurable quality difference", per-item verdicts identical on 94.7% of 622 shared cases, McNemar p ~ 0.30. These runs are eval v1 and are not in the run log.
• Q4_K_M cost nothing in speed over Q8_0 on this model, which was first misread as architecture, then explained as format. Decode was 31.74 against 31.60 t/s (llama-bench, 2026-08-23); the assistant called the dequant-cost explanation wrong and blamed a "per-token floor" from the 24 linear-attention layers. On 2026-09-11 a Q4_0 of the same weights decoded at 43.84-45.4 t/s, so the 31 t/s plateau belonged to the K-quant, not the architecture (Byte alignment, not bit count, governs decode speed).
• Byte alignment, not bit width, set decode speed. Same weights requantised four ways from the Q8_0: Q4_0 45.4, Q5_0 30.5, Q6_K 32.1, Q8_0 31.4 tok/s; repeated on a second prompt within 0.3%. llama-bench independently: Q4_0 44.59 +/- 0.01 against Q4_K_M 31.45 +/- 0.03, build c32d1dabe (10893).
• Runtime does not move throughput; format does. Q4_0 on llama.cpp 44.11, mlx_lm affine-g64 45.30, rapid-mlx 46.64 tok/s (llama.cpp, mlx_lm and rapid-mlx decode at the same speed; only the format differs). The 21.60 tok/s Q4_K_M figure that finding uses looks contended; see Incidents in Qwen3.5-9B Q4_K_M (unsloth GGUF).
• The format result did not transfer to MLX quality-wise, and the 9B's nullschema scores are mostly a phrasing artefact. GGUF servers wrote "not mentioned" where MLX wrote empty strings, and the grader scores the former as fabricated: Q4_K_M 23.6%, Q4_0 6.9%, MLX g64 59.7%, g32 45.8%, mxfp4 34.7% (Runtime changes HOW a model phrases a refusal, and nullschema scores that as fabrication, The GGUF format result does not transfer to MLX -- symmetric quantisation is the WORST there). The assistant's own verdict: "the 9B doesn't get a vote". Treat every nullschema row of this family as unreliable for capability.
• Thinking mode doubles the cost of the study. ~6 hours projected per thinking-on run against ~1 hour thinking off; the thinking-on Q8 run was paused at 18 of 624 rows and never resumed.
• On agentic tool use it was joint-first among the small models once timeouts were excluded: 103 pass / 26 fail / 14 timeout of 143 Toolery scenarios, 72.0% headline and 79.8% excluding timeouts, level with gemma4-e4b at 79.6%. Toolery is outside the run log.
• On grounding it invented little but misread a lot: 8 invented, 22 misread, 47 abstained of 125.
• Naive ternarisation destroys it; mechanically it works. Absmean-ternarised weights packed into TQ2_0 ran at 50.5 tok/s, but perplexity went from 8.78 (Q4_0) to 51,702. The packing is lossless by the kernel's arithmetic; the exact round-trip check on the first test file produced no output, so exactness itself was not demonstrated.

• Alan, 2026-08-23 10:10Z, opening the session: "what qwen 3.8 dense models are available?". The assistant found that 3.8 has exactly one dense model, the 27B, and that "no 0.6B/4B/8B/9B tier" shipped.
• Alan, 10:12Z: "we've already done a load of tests does 3.6 have a 9b?". It does not; the most recent dense 9B Qwen is in 3.5.
• Alan, 10:16Z: "ffs was hoping to test this (you can see in okf what we've been doing), can you pull the 3.5 9b and put it through its paces at q8 and then q4? also want to try gemma4 smaller models at full context". That is the whole brief: a smaller dense Qwen to run Q8 against Q4 on the 624-task study harness, alongside the Gemma 4 small-model long-context work.
• At 10:22Z the assistant asked whether to also pull unsloth's MTP build ("identical weights plus the nextn draft head"), because the fleet notes argued a dense 9B was the one case where speculation should finally pay. Alan's answer was "Yes, both quants with MTP".
• Alan, 10:43Z: "I want to test the 9b models first, there's an update in okf with gpt suggestions on improvements to the test suite".
• Later in its life it stopped being a candidate and became the subject of the format experiments: on 2026-09-11 it was the dense model the Q4_0-vs-Q4_K_M speed lever, the byte-alignment sweep and the MLX-format experiments were run on, because its safetensors source converts quickly and downloads cheaply (assistant). Alan's trigger that day was "we have a lever, chase it, measure it, prove it"; the 9B was one of the five models in the Q4_0 sweep.
• It was never a seat candidate. The 2026-08-24 write-up put it forward as one half of a "single model: qwen3.5-9b-q8" option and Alan moved on to other models (Devstral Small 2 was the next one pulled, Qwen3-Coder 30B was suggested).

• Does Q4_0 hurt or help this model's abstention once the decline-phrasing confound is removed? The post-hoc analyser benchmarks/tools/decline_forms.py exists (Runtime changes HOW a model phrases a refusal, and nullschema scores that as fabrication); the five nullschema rows were never re-scored with it in the run log.
• The MTP speculation A/B on the matched pair was never run. Both MTP builds are gone; the question stays open for any other Qwen3.5-family model.
• A proper quality comparison across quant formats "needs a BF16 source". All four of the quantlab Q4_0/Q5_0/Q6_K/Q8_0 files were re-rounded from the Q8_0, not the BF16, so they measure speed only.
• Why the Q4_K_M measured 21.60 tok/s at tg128 on 2026-09-11 14:44Z while every other measurement of the same weights was 31.2-32.8.
• Eval-v1 vs summary.json: the working session shows ifeval 169/199 (Q8) and 165/198 (Q4) on 2026-08-23; eval/results/summary.json (rewritten 2026-10-09) shows 179 and 175. Which grader state is right is not on record.

Derivatives held elsewhere as their own families: DavidAU's "Fable Heretic" 9B fine-tune (IQ4_NL) appears only in the 2026-08-23 llama-bench table as a same-architecture comparison; it is not covered here.