FreshContextA context lab · Fife
Drawing
02 · Benchmarks
Sheet
2 of 3
Rev
A · 2026-10-01

Model← Models

BTL-4 Compact (IQ2_XXS)

35.1B mixture of experts, 2.1B active, IQ2_XXS (2.30 bpw), 10.0 GB.

Parameters
35.1B / 2.1B active
Quant
IQ2_XXS (2.30 bpw)
Size
10.0 GB
Family
btl
RecordStatus · runtime · seat

btl-4

RejectedMoEresidentllama.cpp

Model card ↗

Outcome as of 20 September 2026, from the seat registry. The write-up below was made when the model was tested.

Why it was tested

The only build badtheorylabs publish. Tested to see what an aggressive IQ codebook costs on a sparse MoE.

What came of it

Rejected in favour of the Q4_K_M build. Same weights, and the quantisation alone cost 27 points of worldview and 17 of toolcall, plus 2.06x throughput (23.6 vs 48.5 tok/s). Evidence for iq-codebook-penalty-transfers-to-moe-kquant-does-not.

Fine-tuned from BTL-4 (Q4_K_M).

Recorded runs14 scored · 95% intervals
SuiteScore95% interval PassedRuntime · profileRecorded
chainwalk0.0%0.0–65.8Passed 0/2llama.cppagentic-fast2026-09-15
contract64.6%50.4–76.6Passed 31/48llama.cppextraction2026-09-15
enrichment87.0%75.6–93.6Passed 47/54llama.cppextraction2026-09-15
epistemics100.0%92.6–100.0Passed 48/48llama.cppstudy-nothink2026-09-15
grounding76.8%68.7–83.3Passed 96/125llama.cppextraction2026-09-15
nullschema36.1%26.0–47.6Passed 26/72llama.cppextraction2026-09-15
screening95.0%86.3–98.3Passed 57/60llama.cppextraction2026-09-15
synthesis80.0%68.2–88.2Passed 48/60llama.cppextraction2026-09-15
tagger85.0%79.9–89.0Passed 204/240llama.cppextraction2026-09-15
tagger_nullable91.7%87.5–94.5Passed 220/240llama.cppextraction2026-09-15
toolcall76.7%64.6–85.6Passed 46/60llama.cppextraction2026-09-15
tools100.0%79.6–100.0Passed 15/15llama.cppagentic-fast2026-09-15
toolscale100.0%91.2–100.0Passed 40/40llama.cppagentic-fast2026-09-15
worldview62.5%47.0–75.8Passed 25/40llama.cppextraction2026-09-15

Runs marked unenforced were measured through mlx_lm on a suite that requires constrained decoding. mlx_lm accepts the schema and ignores it, so those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.

Every run log entry for this model (14)
FinishedSuiteRuntime · profileScoreCoverageGrader · dataset · config
tagger_nullabletagger-nullable/1not recordedextraction91.7%Coverage 240/240g 04e875431e26c4b5d db3addf23a2bc4cec f9c64867e9dfb91b
taggertagger/2not recordedextraction85.0%Coverage 240/240g 63e04d139a0e5ed3d 7ef587bf33e5788ac 81d3e449bf19eaea
enrichmentenrichment/3not recordedextraction87.0%Coverage 54/54g b51222c2ca38cfdfd 75361864baa727abc 0b71856ba217c91c
contractcontract/2not recordedextraction64.6%Coverage 48/48g cc2d23c5bae45474d 1fe53d30cb5480f6c bf1f8546a0e1b97c
chainwalkchainwalk/3not recordedagentic-fast0.0%Coverage 2/2g f4bfcf6f040b0978d 129a9e74408c8919c 8dbbe23a8427e3bc
epistemicsepistemics/4not recordedstudy-nothink100.0%Coverage 48/48g acf84972a0ad5f38d 4bce453fd376553cc 438aa2f62c6153af
groundinggrounding/1not recordedextraction76.8%Coverage 125/125g 30b933dcc6a7e2d4d dcb396faf8b83b0ac 7d9f8293fd874d62
screeningscreening/1not recordedextraction95.0%Coverage 60/60g 0661b49a10be2d2ad 64c97ca2269c29afc 8197a1913a41df95
synthesissynthesis/2not recordedextraction80.0%Coverage 60/60g 4156dd3c13361e8bd 267d1954e225c5b6c 11c6d6217d23f0b7
worldviewworldview/2not recordedextraction62.5%Coverage 40/40g 9acfba1b704feb28d e97835de9e6fc227c 0104404569f7c535
nullschemanullschema/1not recordedextraction36.1%Coverage 72/72g d7448e0f5fc3f9afd 3b91280bb40372efc 9b2dcb073c569fce
toolstools/1not recordedagentic-fast100.0%Coverage 15/15g 2763675a5c3da564d 33a44812f0ad4c39c e43140ad3e77d008
toolscaletoolscale/1not recordedagentic-fast100.0%Coverage 40/40g a2bc3751abfa28e0d b24abb108c409178c 962089edc90681de
toolcalltoolcall/1not recordedextraction76.7%Coverage 60/60g 804e8fc9457649d8d 10af88c170513155c 7d3d7741437436e1
FamilyOthers tested from the btl family
  • BTL-4 (Q4_K_M)Measured, not seated

    35.1B · MoE · Q4_K_M (community conversion) · 20.5 GB

    Fine-tuned from Ornith-1.0-35B on an execution-gated corpus: trajectories kept only where the code ran and passed. 256 experts, 8 routed, 262k native context, ~20 KB/token KV.