FreshContextA context lab · Fife
Drawing
02 · Benchmarks
Sheet
2 of 3
Rev
A · 2026-10-01

Model← Models

K2-Horizon-3.7B (IFM)

3.7B dense, Q4_K_M, 2.9 GB.

Parameters
3.7B
Quant
Q4_K_M
Size
2.9 GB
RecordStatus · runtime · seat

k2-horizon-3.7b

Seated · Dispatchdenseresidentllama.cpp (k2horizon fork)

Model card ↗

Outcome as of 20 September 2026, from the seat registry. The write-up below was made when the model was tested.

Why it was tested

The smallest model measured that perceives tool need. Held at three precisions (Q4/Q8/BF16) which made it the subject of the precision ladder.

What came of it

100% on both tool suites at 2.9 GB - better than its own 36B sibling on toolscale (100% vs 87.5%). Weak downstream: worldview 52.5%, nullschema 38.9%, synthesis 50.0%. The router seat, provided every escalation is a named tool call rather than a judgement. Its precision ladder established that Q8_0 and BF16 differ on 0 of 72 cases.

Recorded runs6 scored · 95% intervals
SuiteScore95% interval PassedRuntime · profileRecorded
chainwalk50.0%9.5–90.5Passed 1/2llama.cppextraction2026-09-09
nullschema43.1%32.3–54.6Passed 31/72llama.cppextraction2026-09-09
screening100.0%94.0–100.0Passed 60/60llama.cppextraction2026-09-09
toolcall100.0%94.0–100.0Passed 60/60llama.cppextraction2026-09-09
tools86.7%62.1–96.3Passed 13/15llama.cppextraction2026-09-09
toolscale100.0%91.2–100.0Passed 40/40llama.cppextraction2026-09-09

Runs marked unenforced were measured through mlx_lm on a suite that requires constrained decoding. mlx_lm accepts the schema and ignores it, so those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.

Every run log entry for this model (6)
FinishedSuiteRuntime · profileScoreCoverageGrader · dataset · config
chainwalkchainwalk/3not recordedextraction50.0%Coverage 2/2g f4bfcf6f040b0978d 129a9e74408c8919c 497e8e74501f9a9e
nullschemanullschema/1not recordedextraction43.1%Coverage 72/72g d7448e0f5fc3f9afd 3b91280bb40372efc c1cd134c67de1621
screeningscreening/1not recordedextraction100.0%Coverage 60/60g 0661b49a10be2d2ad 64c97ca2269c29afc 5d3e0bfbb30467e0
toolstools/1not recordedextraction86.7%Coverage 15/15g 2763675a5c3da564d 33a44812f0ad4c39c 4b2e40f5222213ec
toolscaletoolscale/1not recordedextraction100.0%Coverage 40/40g a2bc3751abfa28e0d b24abb108c409178c 4535019c467a8444
toolcalltoolcall/1not recordedextraction100.0%Coverage 60/60g 804e8fc9457649d8d 10af88c170513155c f848b35c571d582e
FamilyOthers tested from the k2-horizon family
  • K2-Horizon-36B-A4B (IFM)Seated · agent

    36B · MoE · Q4_K_M · 20.8 GB

    MoVA (Mixture-of-Value Attention) extends sparsity into the value component of attention. 4B active gives it 25 tok/s at 5k where a dense 27B manages 8.6. Ties the top of the board on tool invocation and holds a 200-step chain without deviating.