FreshContextA context lab · Fife
Drawing
02 · Benchmarks
Sheet
2 of 3
Rev
A · 2026-10-01

K2-Horizon-36B-A4B · file← K2-Horizon-36B-A4B

Q4_1

A format experiment: one test, then deleted. Requantised here from the local Q4_K_M.

Size
23.6 GB
Status
Deleted 11 Sep 2026
Scored runs
1
Smallest Mac
48 GB estimate, 8K context
Results1 scored · 95% intervals
SuiteScore95% interval PassedRuntime · profileRecorded
nullschemak2-36b-Q4_150.0%38.7–61.3Passed 36/72llama.cppextraction2026-09-11

Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.

The recordHow it was made · how it was served · what went wrong · what happened to it

Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.

FileK2-36B__Q4_1.gguf
FormatQ4_1, ~21.95 GiB
Where frommade here 2026-09-11 from the Q4_K_M
Statusdeleted 2026-09-11

How it was made

  • Command:
llama.cpp-k2horizon/build/bin/llama-quantize --allow-requantize \
K2-Horizon-36B-A4B-Q4_K_M.gguf models/_q40test/K2-36B__Q4_1.gguf Q4_1
  • Tool: k2horizon fork @ 35999d1 (build 20); bench with the fork's llama-bench -p 0 -n 64 -ngl 99 -r 3.
  • Date: 2026-09-11.
  • Size: ~21.95 GiB per llama-bench; 22476.24 MiB tensor data (5.04 BPW); 22G du. Exact bytes never recorded; sha256 not on record.
  • Who asked / why: part of the format ladder that followed Alan's "we have a lever, chase it, measure it, prove it". The hypothesis was written into the script at the time: "Q4_1 has Q4_0's alignment (2 weights/byte) AND Q4_K_M's asymmetry, with exact fp16 scales instead of quantised ones... If alignment drives speed and asymmetry drives quality, this wins both".

How it was served

labeldates (UTC)binary and versionctx / -np / KV / -fa / --jinja / --reasoning-formatharness profilewhy this label exists
k2-36b-Q4_12026-09-11, finished 13:48:25 (run-log 14:48:25 BST)fork llama-server, 0.3.0-dev (build 20, commit 35999d1)131072 / 1 / q8_0 / on / not on the line / deepseekextraction; --model-id = the file path; nullschema onlybyte-aligned and asymmetric: "If alignment drives speed and asymmetry drives quality, this wins both"

Seats held: none.

Run-log runtime: llama.cpp b20 (35999d1) (runtime_source "props"). Never seated.

Decisions

  • Made only to test a mechanism. It was never a serving candidate.
  • Outcome: Q4_1 falsified the "exact fp16 scales" explanation for Q4_0's nullschema gain. The assistant then offered symmetric quantisation as a hypothesis "I'd hold... loosely". Finding Quantisation format changes output CONVENTIONS, not abstention judgement (revised) later attributed the gain to fewer malformed responses, not judgement.
  • Deleted 2026-09-11 15:14Z.

What went wrong

  • Its result "kills my mechanism": the fp16-scale explanation for the Q4_0 gain was withdrawn.
  • The run-log llama_build field names upstream build 10893 rather than the fork (same issue as every K2 entry).
  • The manifest records the full _q40test path as the model id; that directory no longer exists.
  • Missing flag (standing check): the server line had no --jinja; build 20's default is not on record. Partial runs: none (72 rows).

What happened to it

Requantised here 2026-09-11 → nullschema + speed 2026-09-11 → deleted 2026-09-11 15:14Z. No RESTORE-deleted-*.sh entry. Re-creatable only via BF16 → Q4_K_M → Q4_1 (K2-Horizon-36B-A4B Q4_K_M (quantised here from IFM's BF16) + the command above). The run-log result survives.