K2-Horizon-36B-A4B · file← K2-Horizon-36B-A4B
Q8_0
A larger, more precise copy, to see what precision buys. Quantised here from IFM's BF16.
- Size
- 39.8 GB
- Status
- Deleted 12 Sep 2026
- Scored runs
- 2
- Smallest Mac
- 64 GB estimate, 8K context
| Suite | Score | 95% interval | Passed | Runtime · profile | Recorded |
|---|---|---|---|---|---|
| nullschemak2-36b-q8-precision | 70.8% | 59.5–80.1 | Passed 51/72 | llama.cppextraction | 2026-09-10 |
| worldviewk2-36b-q8-precision | 55.0% | 39.8–69.3 | Passed 22/40 | llama.cppextraction | 2026-09-10 |
Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.
Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.
| File | K2-Horizon-36B-A4B-Q8_0.gguf |
| Format | Q8_0, 39,831,174,496 B |
| Where from | made here 2026-09-09 from the BF16 |
| Status | deleted 2026-09-12 08:04Z |
| Hugging Face | as above, for Q8_0 |
How it was made
- Input: the same downloaded
K2-Horizon-36B-BF16.gguf(74,924,627,296 B,IFM/K2-Horizon-MoVA-36B-A4B-GGUF). - Command:
llama.cpp-k2horizon/build/bin/llama-quantize K2-Horizon-36B-BF16.gguf K2-Horizon-36B-A4B-Q8_0.gguf Q8_0- Tool: k2horizon fork @
35999d1(build 20). - Date: 2026-09-09 (ls mtime "9 Sep 17:01" local).
- Size: 39,831,174,496 B. sha256 not on record.
- Who asked / why: Alan: "we could quantize q8 and run it, see what it can do against the gauntlet". In practice the Q4_K_M was seated first, because 37.1 GB needed the other servers stood down.
- Hugging Face: same byte count as IFM's later-published Q8_0; not downloaded from there.
How it was served
| label | dates (UTC) | binary and version | ctx / -np / KV / -fa / --jinja / --reasoning-format | harness profile | why this label exists |
|---|---|---|---|---|---|
k2-36b-q8-precision | 2026-09-10 12:00–12:02 | fork llama-server, 0.3.0-dev (build 20, commit 35999d1) | 32768 / 1 / q8_0 / on / on / deepseek | extraction | the 36B rung of the precision ladder after Alan's "whether the fp16 really nails things". Only nullschema and worldview ran, by design ("the suite carrying the finding") |
Seats held: none.
Also the source file for K2-Horizon-36B-A4B Q4_0 (requantised here from the local Q8_0, one extra rounding pass) and K2-Horizon-36B-A4B Q4_0 with Q8_0 output and token-embedding tensors (requantised here from the local Q8_0) on 2026-09-11. Run-log runtime: llama.cpp b20 (35999d1), k2horizon fork.
Decisions
- Kept on disk after the precision run as the Q8_0 reference build and used as the one-rounding source for the clean Q4_0 attempt.
- Deleted 2026-09-12 08:04Z in the "Q8_0 reference builds" group on Alan's "kill and q8 and q4_k_m safely?".
- Never seated: 37 GB plus KV did not leave room for co-resident seats (reconstructed).
What went wrong
- Speed measurement lost: a sed error swallowed the Q8_0 decode speed, so it was never recorded.
- The run-log
llama_buildfield names upstream build 10893 rather than the fork (same issue as every K2 entry). - Standing checks: both results files are complete (72 and 40 rows); the server line had
--jinjaand--reasoning-format deepseek.
What happened to it
Quantised here 2026-09-09 → benched 2026-09-10 → source for two Q4_0 experiments 2026-09-11 → deleted 2026-09-12 08:04Z. No RESTORE-deleted-*.sh entry. Re-create from IFM's BF16 with the command above.