FreshContextA context lab · Fife
Drawing
02 · Benchmarks
Sheet
2 of 3
Rev
A · 2026-10-01

Model← Models

Bonsai-27B (Q1_0)

26.9B dense, Q1_0 (1.13 bpw), 3.5 GB.

Parameters
26.9B
Quant
Q1_0 (1.13 bpw)
Size
3.5 GB
Family
bonsai
RecordStatus · runtime · seat

bonsai-27b-1bit

Rejecteddenseresidentllama.cpp (PrismML fork)

Model card ↗

Outcome as of 20 September 2026, from the seat registry. The write-up below was made when the model was tested.

Why it was tested

Same base as the ternary build at a more aggressive operating point. Tested to find which capabilities survive 1-bit.

What came of it

Keeps tool use perfectly (toolcall 100.0, tools and toolscale 100.0) and loses structured compliance: contract 75.0 to 25.0, synthesis 66.7 to 35.0. Abstention also falls, 27.8 to 18.1. Faster than the ternary build (20.0 vs 17.3) despite the same parameter count. Three suites IMPROVED, which is unexplained and should not be trusted without a repeat.

Fine-tuned from Ternary-Bonsai-27B (PQ2_0).

Recorded runs14 scored · 95% intervals
SuiteScore95% interval PassedRuntime · profileRecorded
chainwalk0.0%0.0–65.8Passed 0/2llama.cppagentic-fast2026-09-15
contract25.0%14.9–38.8Passed 12/48llama.cppextraction2026-09-15
enrichment79.6%67.1–88.2Passed 43/54llama.cppextraction2026-09-15
epistemics87.5%75.3–94.1Passed 42/48llama.cppstudy-nothink2026-09-14
grounding67.2%58.6–74.8Passed 84/125llama.cppextraction2026-09-14
nullschema18.1%10.9–28.5Passed 13/72llama.cppextraction2026-09-14
screening96.7%88.6–99.1Passed 58/60llama.cppextraction2026-09-14
synthesis35.0%24.2–47.6Passed 21/60llama.cppextraction2026-09-14
tagger85.4%80.4–89.3Passed 205/240llama.cppextraction2026-09-15
tagger_nullable90.8%86.5–93.9Passed 218/240llama.cppextraction2026-09-15
toolcall100.0%94.0–100.0Passed 60/60llama.cppextraction2026-09-14
tools100.0%79.6–100.0Passed 15/15llama.cppagentic-fast2026-09-14
toolscale100.0%91.2–100.0Passed 40/40llama.cppagentic-fast2026-09-14
worldview40.0%26.3–55.4Passed 16/40llama.cppextraction2026-09-14

Runs marked unenforced were measured through mlx_lm on a suite that requires constrained decoding. mlx_lm accepts the schema and ignores it, so those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.

Every run log entry for this model (14)
FinishedSuiteRuntime · profileScoreCoverageGrader · dataset · config
tagger_nullabletagger-nullable/1not recordedextraction90.8%Coverage 240/240g 04e875431e26c4b5d db3addf23a2bc4cec f3c4a61093079a3e
taggertagger/2not recordedextraction85.4%Coverage 240/240g 63e04d139a0e5ed3d 7ef587bf33e5788ac d75dadae6c07b420
enrichmentenrichment/3not recordedextraction79.6%Coverage 54/54g b51222c2ca38cfdfd 75361864baa727abc 612035dbfdcf9143
contractcontract/2not recordedextraction25.0%Coverage 48/48g cc2d23c5bae45474d 1fe53d30cb5480f6c f3e9344f382b675d
chainwalkchainwalk/3not recordedagentic-fast0.0%Coverage 2/2g f4bfcf6f040b0978d 129a9e74408c8919c 315259486d2b1e0e
epistemicsepistemics/4not recordedstudy-nothink87.5%Coverage 48/48g acf84972a0ad5f38d 4bce453fd376553cc 2f1d2e8b3a422ec6
groundinggrounding/1not recordedextraction67.2%Coverage 125/125g 30b933dcc6a7e2d4d dcb396faf8b83b0ac 0f1cdb00af0adb60
screeningscreening/1not recordedextraction96.7%Coverage 60/60g 0661b49a10be2d2ad 64c97ca2269c29afc 06fe499419357acb
synthesissynthesis/2not recordedextraction35.0%Coverage 60/60g 4156dd3c13361e8bd 267d1954e225c5b6c b6b3f7a5739fbaf7
worldviewworldview/2not recordedextraction40.0%Coverage 40/40g 9acfba1b704feb28d e97835de9e6fc227c c66967b2cb84959c
nullschemanullschema/1not recordedextraction18.1%Coverage 72/72g d7448e0f5fc3f9afd 3b91280bb40372efc b6bf04bcfeb45185
toolstools/1not recordedagentic-fast100.0%Coverage 15/15g 2763675a5c3da564d 33a44812f0ad4c39c a9cb56f68c317ce8
toolscaletoolscale/1not recordedagentic-fast100.0%Coverage 40/40g a2bc3751abfa28e0d b24abb108c409178c 8ee047a8f4681484
toolcalltoolcall/1not recordedextraction100.0%Coverage 60/60g 804e8fc9457649d8d 10af88c170513155c 0f6815eef83b64d0
FamilyOthers tested from the bonsai family
  • Bonsai-8B (Q1_0)Rejected

    8.0B · dense · Q1_0 (1.13 bpw) · 1.1 GB

    Derived from Qwen3-8B, not the Qwen3.6-27B backbone the larger Bonsai models use. Tested as the small corner of a size-by-bit-width grid.

  • Ternary-Bonsai-27B (PQ2_0)Measured, not seated

    26.9B · dense · PQ2_0 (2.13 bpw, group-128) · 7.2 GB

    prism-ml's group-128 ternary pack, built for their own llama.cpp fork's hybrid-attention kernels. The upstream Q2_g64 variant lacks them.

  • Ternary-Bonsai-27B (prism-ml)Measured, not seated

    27B · dense · Q2_g64 (1.71 bits/weight) · 7.1 GB

    End-to-end ternary weights on a Qwen3.6-27B backbone with ~75% linear attention, claiming 262K context and 94.6% of FP16 intelligence retained. 11.7 GB resident at a 128k window - a third of K2-36B for the same context.