K2-Horizon-36B-A4B · file← K2-Horizon-36B-A4B
Q4_0 with Q8_0 output and token-embedding tensors
A Q4_0 that keeps two layers at higher precision. Never scored, for the same reason. Requantised here from the local Q8_0.
- Size
- 21.7 GB
- Status
- Deleted 11 Sep 2026
- Scored runs
- 0
ResultsNever scored
This file never completed a scored run. Its record below says why.
The recordHow it was made · how it was served · what went wrong · what happened to it
Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.
| File | K2-36B__dynQ4_0.gguf |
| Format | Q4_0 with Q8_0 output and token-embedding tensors, from the Q8_0, 20737.43 MiB (4.65 BPW) |
| Where from | made here 2026-09-11 |
| Status | deleted 2026-09-11 15:51Z |
How it was made
- Input:
models/k2-horizon-36b-a4b/K2-Horizon-36B-A4B-Q8_0.gguf(39,831,174,496 B; see K2-Horizon-36B-A4B Q8_0 (quantised here from IFM's BF16)). This is one rounding pass beyond Q8_0 (BF16 → Q8_0 → Q4_0), against two for K2-Horizon-36B-A4B Q4_0 (requantised here from the local Q4_K_M). - Command:
llama.cpp-k2horizon/build/bin/llama-quantize --allow-requantize --output-tensor-type q8_0 --token-embedding-type q8_0 K2-Horizon-36B-A4B-Q8_0.gguf models/_q40test/K2-36B__dynQ4_0.gguf Q4_0- Tool: k2horizon fork @
35999d1(build 20). - Date: 2026-09-11, ~14:14–14:44Z.
- Size: 20737.43 MiB tensor data (4.65 BPW), 20.25 GiB per llama-bench;
du -h20G. Exact bytes and sha256 not on record. - Who asked / why: Alan, 14:12Z: "should we get the base model and requant to q4_0". The assistant used the Q8_0 already on disk rather than re-downloading the BF16. Script header: "B Q8_0 -> Q4_0 + Q8_0 output/embed byte-aligned DYNAMIC: UD's idea without UD's K-quant alignment penalty"
How it was served
| label | dates (UTC) | binary and version | ctx / -np / KV / -fa / --jinja / --reasoning-format | harness profile | why this label exists |
|---|---|---|---|---|---|
| — (no run-log label) | 2026-09-11 (attempted only) | fork llama-server, 0.3.0-dev (build 20, commit 35999d1) | 131072 / 1 / q8_0 / on / not on the line / deepseek | extraction (nullschema attempted) | the run aborted at the harness schema probe before any case was written, so no label exists |
Seats held: none.
Decisions
- Made to test whether a cleaner Q4_0 beats the double-rounded one on nullschema.
- Deleted unscored 2026-09-11 15:51Z, under Alan's "you can do 1 - 5". The deletion line read "removed 40 GB of unvalidated variants", one of the two Q8-sourced builds. Earlier, at 15:14Z, both had been kept through the disk sweep ("keep: …") under Alan's "keep any q4_0 and drafters".
- Not published.
What went wrong
- Harness schema-probe abort. nullschema died before writing a case: "RuntimeError: schema probe returned only reasoning_content and no content -- the model is still thinking. Raise max_tokens". The double-rounded Q4_0 passed the same probe with the same flags. That made this a provenance-dependent behaviour difference the session could not explain. It cost the only clean-vs-double-rounded quality comparison; nothing fixed it before deletion.
- Missing flag (standing check): the server line had no
--jinja. Whether that contributed to the probe failure is not on record.
What happened to it
Requantised here 2026-09-11 → speed-benched → quality run aborted → deleted 2026-09-11 15:51Z. No RESTORE-deleted-*.sh entry. Re-creatable from a Q8_0 made from IFM's BF16 (K2-Horizon-36B-A4B BF16 GGUF (downloaded from IFM, rev c8dde8bc) → K2-Horizon-36B-A4B Q8_0 (quantised here from IFM's BF16) → the command above).