Q4_K_M

• Build of llama.cpp on 5 runs was 0.4.0-dev (build 10893, commit c32d1dabe), now 0.2.0-dev (build 10618, commit eb25b7263). The runs finished before llama.cpp was upgraded that day, so the serving build was b10618; the stamp was read after the upgrade.

  Corrected 24 October 2026 · a mechanical fix

Where it came from

• Command: a 6-connection range download (pget); the exact invocation is not on record (the 2026-08-11 session is not on record; the later fetch scripts use $SP/pget.sh <url> <file> 6, e.g. for another model).
• Who asked / why: part of the runtime-comparison setup; Alan's words not on record.
• Revision: not on record.

How it was served

Command (started 10:32:47Z per the script's own banner):

$BIN --model models/qwen3-8b-gguf/Qwen3-8B-Q4_K_M.gguf --jinja \
     --reasoning-format deepseek -c 32768 -np 1 -fa on -ctk q8_0 -ctv q8_0 -ngl 999 --alias qwen3-8b
python harness/runner.py --suite <s> --model <server> --model-id qwen3-8b \
     --label qwen3-8b-midrange --profile extraction --no-resume      # s = toolcall toolscale worldview nullschema synthesis

Footprint 7.1 GB. The server was stopped between models by pkill -f "port <port>".

Other serving of this file, outside the run log (reconstructed from scripts and logs):

• Eval v1 budget run, Aug 12: llama-server, n_slots = 12, n_ctx_slot = 5120 (eval/results_budget3500/qwen3-8b.server.log), eval/runner.py MODELS entry qwen3-8b.
• Concurrency test (eval/conc_llamacpp.py, file mtime 2026-08-12 12:15): -ngl 99 -c 2048*slots -np {1,8,16} -cb --jinja, 32 prompts, max_tokens 200, temperature 0, seed 42.
• llama-bench 2026-08-23 13:06Z on b10369 (6e62ba538), -ngl 99, defaults: pp512 384.2, tg128 41.08 (eval/results_fleet_bench.json).
• PPL 2026-08-30 17:51Z+: llama-cpp-turboquant/build/bin/llama-perplexity -m... -f eval/ppl-corpus.txt -c 4096 --chunks 12 over five KV types.
• Router preset [qwen3-8b] (on-demand, --models-max 1): registered on 2026-09-08 and 2026-09-09 until the preset was moved to the Q4_0 on 2026-09-11 22:28Z.

Decisions

• Kept as the router's cheap backend and as the only Qwen3-8B build after the Q8_0 duplicate was removed.
• Benched on the agentic board only because Alan remembered it ("The qwen8b model was very good.").
• Superseded by the Q4_0 requant on 2026-09-11 (router preset) and deleted 2026-09-12 08:04:52Z: Alan, 08:03Z, "We can do some more cleanup too, kill and q8 and q4_k_m safely?". The assistant's classifier marked it "SAFE — faster sibling present, not serving" and deleted it with four other K-quants.
• Not published to HF (it is someone else's file).

What went wrong

• Stale build stamp (standing check). The run log's llama_build for all five qwen3-8b-midrange entries said 0.4.0-dev (build 10893, commit c32d1dabe), while runtime_version said b10618 (eb25b7263). The runs finished 10:34-10:46Z; upgrade_llamacpp.sh started at 12:17:49Z and the reflog pull was 13:17:59 +0100 = 12:17:59Z. So the binary that served them was b10618 and the 10893 stamp was read after the upgrade. The evidence-based runtime_version was right; llama_build was wrong.
• The HumanEval claim. "HumanEval 86.8% strict, beating qwen3.6-35b-a3b" (a lab note, the harness runbook) is the first 106 tasks, which are ordered easy-first; on all 164 it is 78.7% against the 35B's 79.9%. The runbook's own rule 3 ("Never validate a budget on the first N items of a benchmark") describes the trap. The two harness notes were revised on 2026-10-05 but the qwen3-8b row was left as "qwen3-8b (partial)".
• Mis-framed first reading. The assistant first read toolcall 0/60 as "a training property, not a size property", then 4 minutes later, after toolscale returned 40/40 with finish_reason: tool_calls, corrected it. No data was wrong.
• Missing flags (standing check): --jinja and --reasoning-format deepseek present; how the profile's thinking: false reached the Qwen3 template (chat_template_kwargs or default) is not on record.
• Partial runs: none.
• Doc claim: the lab note says BF16 is "worth keeping"; see Qwen3-8B BF16 safetensors (mlx-community).

What happened to it

Downloaded 2026-08-11 -> eval v1 probes, runtime study, llama-bench -> router preset [qwen3-8b] -> PPL subject 2026-08-30 -> benched 2026-09-10 (qwen3-8b-midrange) -> requantised to Q4_0 2026-09-11 13:19Z -> router preset moved 2026-09-11 22:28Z -> deleted 2026-09-12 08:04:52Z (no RESTORE entry; RESTORE-deleted-20260911.sh and later scripts do not mention it) -> re-downloadable from unsloth/Qwen3-8B-GGUF (inferred; current revision may differ). A local Q4_0 with the same lineage remains (Qwen3-8B Q4_0 (requantised here from the local Q4_K_M)).