FreshContextA context lab · Fife
Drawing
02 · Benchmarks
Sheet
2 of 3
Rev
A · 2026-10-01

Model← Models

BTL-4 (Q4_K_M)

35.1B mixture of experts, 2.1B active, Q4_K_M (community conversion), 20.5 GB.

Parameters
35.1B / 2.1B active
Quant
Q4_K_M (community conversion)
Size
20.5 GB
Family
btl
RecordStatus · runtime · seat

btl-4-q4km

Measured, not seatedMoEresidentllama.cpp

Model card ↗

Outcome as of 20 September 2026, from the seat registry. The write-up below was made when the model was tested.

Why it was tested

Fine-tuned from Ornith-1.0-35B on an execution-gated corpus: trajectories kept only where the code ran and passed. 256 experts, 8 routed, 262k native context, ~20 KB/token KV.

What came of it

Best grounder and abstainer measured. nullschema 44.4% is the highest on this board by 8 points; epistemics 100%, enrichment 98.8%, grounding 76.8%. Beats ornith on toolcall (93.3 vs 80.0) and contract (70.8 vs 66.7) at 48.5 tok/s in 20.5 GB. NOT a driver replacement: worldview 55.0 against ornith's 85.0, and 0/8 on the superseded arm where ornith scores 8/8. It knows when it has nothing; it does not know when what it has is out of date.

Derived from this: BTL-4 Compact (IQ2_XXS).

Recorded runs14 scored · 95% intervals
SuiteScore95% interval PassedRuntime · profileRecorded
chainwalk0.0%0.0–65.8Passed 0/2llama.cppagentic-fast2026-09-15
contract70.8%56.8–81.8Passed 34/48llama.cppextraction2026-09-15
enrichment81.5%69.2–89.6Passed 44/54llama.cppextraction2026-09-15
epistemics100.0%92.6–100.0Passed 48/48llama.cppstudy-nothink2026-09-15
grounding76.8%68.7–83.3Passed 96/125llama.cppextraction2026-09-15
nullschema44.4%33.5–55.9Passed 32/72llama.cppextraction2026-09-15
screening91.7%81.9–96.4Passed 55/60llama.cppextraction2026-09-15
synthesis83.3%72.0–90.7Passed 50/60llama.cppextraction2026-09-15
tagger87.1%82.2–90.7Passed 209/240llama.cppextraction2026-09-15
tagger_nullable48.8%42.5–55.0Passed 117/240llama.cppextraction—
toolcall93.3%84.1–97.4Passed 56/60llama.cppextraction2026-09-15
tools100.0%79.6–100.0Passed 15/15llama.cppagentic-fast2026-09-15
toolscale100.0%91.2–100.0Passed 40/40llama.cppagentic-fast2026-09-15
worldview55.0%39.8–69.3Passed 22/40llama.cppextraction2026-09-15

Runs marked unenforced were measured through mlx_lm on a suite that requires constrained decoding. mlx_lm accepts the schema and ignores it, so those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.

Every run log entry for this model (15)
FinishedSuiteRuntime · profileScoreCoverageGrader · dataset · config
tagger_nullabletagger-nullable/1not recordedextraction71.3%Coverage 68.3% coverageg 04e875431e26c4b5d db3addf23a2bc4cec 53f41ca547c1a228
tagger_nullabletagger-nullable/1not recordedextraction71.3%Coverage 164/164g 04e875431e26c4b5d db3addf23a2bc4cec 53f41ca547c1a228
taggertagger/2not recordedextraction87.1%Coverage 240/240g 63e04d139a0e5ed3d 7ef587bf33e5788ac 10dcbcd1b3bb7b91
enrichmentenrichment/3not recordedextraction81.5%Coverage 54/54g b51222c2ca38cfdfd 75361864baa727abc 3b3b53961682ee7f
contractcontract/2not recordedextraction70.8%Coverage 48/48g cc2d23c5bae45474d 1fe53d30cb5480f6c c5f63e89ad62fa86
chainwalkchainwalk/3not recordedagentic-fast0.0%Coverage 2/2g f4bfcf6f040b0978d 129a9e74408c8919c 9b7c71d675210c91
epistemicsepistemics/4not recordedstudy-nothink100.0%Coverage 48/48g acf84972a0ad5f38d 4bce453fd376553cc 0689f579cb5fa875
groundinggrounding/1not recordedextraction76.8%Coverage 125/125g 30b933dcc6a7e2d4d dcb396faf8b83b0ac b6421e71dc1b07d5
screeningscreening/1not recordedextraction91.7%Coverage 60/60g 0661b49a10be2d2ad 64c97ca2269c29afc df6b9c70d3ecb99c
synthesissynthesis/2not recordedextraction83.3%Coverage 60/60g 4156dd3c13361e8bd 267d1954e225c5b6c 34ecdb4bf4ded955
worldviewworldview/2not recordedextraction55.0%Coverage 40/40g 9acfba1b704feb28d e97835de9e6fc227c 7cf929c72ed89ccb
nullschemanullschema/1not recordedextraction44.4%Coverage 72/72g d7448e0f5fc3f9afd 3b91280bb40372efc 43c9232ce793d709
toolstools/1not recordedagentic-fast100.0%Coverage 15/15g 2763675a5c3da564d 33a44812f0ad4c39c a04248bbed30bba3
toolscaletoolscale/1not recordedagentic-fast100.0%Coverage 40/40g a2bc3751abfa28e0d b24abb108c409178c 83afd32cbd70262a
toolcalltoolcall/1not recordedextraction93.3%Coverage 60/60g 804e8fc9457649d8d 10af88c170513155c 0adbf9c03a0169c7
FamilyOthers tested from the btl family
  • BTL-4 Compact (IQ2_XXS)Rejected

    35.1B · MoE · IQ2_XXS (2.30 bpw) · 10.0 GB

    The only build badtheorylabs publish. Tested to see what an aggressive IQ codebook costs on a sparse MoE.