FreshContextA context lab · Fife
Drawing
02 · Benchmarks
Sheet
2 of 3
Rev
A · 2026-10-01

Ternary-Bonsai-2-27B · file← Ternary-Bonsai-2-27B

Ternary-Bonsai-2-27B PTQ1_0

The one we kept: PrismML's smaller 1.75-bit file, with identical answers. Briefly seated on the M4 Pro Mac mini; still on disk. Prism-ml, 1.75 bpw.

Size
5.9 GB
Status
On disk
Scored runs
8
Smallest Mac
16 GB estimate, 8K context
Results8 scored · 95% intervals
SuiteScore95% interval PassedRuntime · profileRecorded
epistemicsbonsai2-27b-ptq1100.0%92.6–100.0Passed 48/48llama.cppstudy-nothink2026-09-18
groundingbonsai2-27b-ptq166.4%57.7–74.1Passed 83/125llama.cppextraction2026-09-18
nullschemabonsai2-27b-ptq131.9%22.3–43.4Passed 23/72llama.cppextraction2026-09-18
screeningbonsai2-27b-ptq198.3%91.1–99.7Passed 59/60llama.cppextraction2026-09-18
synthesisbonsai2-27b-ptq183.3%72.0–90.7Passed 50/60llama.cppextraction2026-09-18
toolcallbonsai2-27b-ptq1100.0%94.0–100.0Passed 60/60llama.cppextraction2026-09-18
toolscalebonsai2-27b-ptq197.5%87.1–99.6Passed 39/40llama.cppagentic-fast2026-09-18
worldviewbonsai2-27b-ptq182.5%68.0–91.3Passed 33/40llama.cppextraction2026-09-18

Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.

Corrections10 to the record published 21 October 2026

  • Build of llama.cpp on 9 runs was 0.4.0-dev (build 10893, commit c32d1dabe), now 0.2.0-dev (build 1, commit 6ac5eb0). Served by the PrismML llama.cpp fork (build 1, commit 6ac5eb0), as every run manifest records; the stamp named the stock binary instead.

    Corrected 24 October 2026 · a mechanical fix

  • On the worldview run of 18 September 2026: voided. A partial run: the gauntlet was stopped with 9 of 40 cases written. The complete 40-case run follows it.

    Corrected 24 October 2026 · a mechanical fix

The recordHow it was made · how it was served · what went wrong · what happened to it

Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.

FileTernary-Bonsai-2-27B-PTQ1_0.gguf, 5,946,648,928 B
FormatGGUF 1.75 bpw
Where fromdownloaded 2026-09-18
Statuson disk (the Mac Studio); M4 Pro seat taken down by 2026-09-20
Hugging Facesame repo

Where it came from

  • Source: prism-ml/Ternary-Bonsai-2-27B-gguf, file Ternary-Bonsai-2-27B-PTQ1_0.gguf, 5,946,648,928 bytes (stat; equals the HF API size). 1.75 bits per weight ternary, group 128 (model_ftype from /props).
  • Downloaded: 2026-09-18, started 07:27Z, complete before 07:31Z, same hf_hub_download recipe as the PQ2_0, into models/bonsai2-27b/. The first look at it was to see whether the drafter pairs with the 1-bit target. Revision and sha256: not on record.
  • Second copy: the M4 Pro Mac mini fetched the same file through prism's Bonsai-demo repo (hf_hub_download into models/bonsai2-gguf/27B) for the permanent seat (a lab note); provenance of that copy is that note.

How it was served

labeldates (UTC)binary and versionctx / -np / KV / -fa / --jinja / --reasoning-formatharness profilewhy this label exists
bonsai2-27b-ptq12026-09-18 07:35Z start; paused 08:07Z (gauntlet killed to free the GPU for a drafter test); resumed 11:31Z; run-log 08:37:51 to 12:47:56 BST (07:37Z to 11:47Z); 8 suitesllama.cpp-prism/build/bin/llama-server, b1-6ac5eb0 (manifests)16384 / 1 / f16 / default / on / not on the line (-ngl 99 --no-warmup --alias bonsai2-ptq1)extraction / agentic-fast / study-nothinkAlan: "What that on the 1bit though?": does the smaller ternary container change anything. Coherence 12/12 at 0.99 first.
(no label) contradiction/1 probe2026-09-18 20:32Zsame binary8192 / 1 / f16 / default / on / not on the line (--alias bonsai2-ptq1)probe (own grader)Test whether edge0 / ornith / bonsai-2 detect conflicting records (Two seats detect temporal conflicts, not one: k2-horizon-36b matches ornith at 4/4).
(no label) edgefill/1 probe2026-09-18 20:42Zsame binarysame flagsprobeTyped-edge extraction without constrained decoding (Every seat emits valid typed edges; three of five turn a hedged future statement into an asserted fact).
(no label) feel-test seat on the Mac Studio2026-09-19 01:14Z to 01:26Zllama.cpp-prism fork8192 / 1 / q4_0 / on / on / not on the line (-ngl 999 --alias bonsai2-27b)noneAlan could not select it in the fleet UI because the seat was down. Stopped when edge0 plus bonsai caused memory pressure.
(no label) M4 Pro permanent seat2026-09-19; taken down by 2026-09-20 17:33Zprism fork prism-b10709-9a9394a, launchd agent16384 / 1 slot (lab note)noneSee a lab note; Alan, 2026-09-20: "bonsai 2 has been taken down on [the M4 Pro Mac mini]".

Seats held: the Mac Studio feel-test seat above; the M4 Pro seat per the lab note (14.5 tok/s, 7.0 GB, 1 slot x 16k).

Decisions

  • Why downloaded: to see whether the 18% smaller 1.75 bpw container costs anything, and whether the DFlash2 drafter would pair with it.
  • Result: identical to the PQ2_0 on every suite, so "PTQ1_0 is the correct file" for any prism ternary-trained checkpoint (PTQ1_0 and PQ2_0 produce byte-identical output on a ternary-trained model, so the smaller file is free).
  • Seat placement (2026-09-18 12:00Z to 12:31Z). Alan: "so... this leaves me with small tool caller (k2 horizon?), the edge0 memory workers and a slower bonsai 2 ternary?". The assistant's sizing: at 128K the KV (9.0 GB) overtakes the 5.5 GB of weights, the seat is 14.5 GB, which blows the Mac Studio's budget alongside ornith, k2-horizon and edge0 (51.6 GB past the 44.7 GB paging threshold). Alan: "bonsai 2 can live elsewhere"; the M4 Pro (24 GB) became its seat.
  • Alan on a multi-box worker idea: "So a 5.5gb worker I could have one on m4 base, one on m4 pro and one on [the Mac Studio] but I also wanted bonsai running so I had qwen 3.8 available.".
  • Feel test (2026-09-19): "it's not letting me select bonsai"; "it's doing 14.2 tps on bonsai, that's totally acceptable"; "it may get authoring jobs / synthesis". The assistant's placement: ornith reasons, k2 calls, edge0 reads, bonsai writes, with a note that Bonsai-2 turns hedged statements into asserted edges 4 of 4 times, so it writes prose and never edges. Memory: "ooft edge + bonsai put a little memory pressure on there"; the assistant stopped it on the Mac Studio.
  • Later: on 2026-09-28 Alan asked whether, with a fast enough runtime, "the bonsai seat possibly changes to either full fat 3.8 q4_0?"; AGENTS.md (2026-10) still lists llama.cpp-prism as the "bonsai2 swap binary (seat is on the M4 Pro for now — judgment call)". The 2026-10-11 sweep kept this file ("PTQ1_0, kept, 12G").

What went wrong

  • Partial worldview row (standing check, caught): two run-log worldview rows exist for this label: 9 of 9 attempted at 09:07 BST (the gauntlet was paused mid-suite), then 33/40 at 12:34 BST. Only the second is the suite; the first reads as a perfect 100.0 with coverage 100%. The resume log line showed 24/31, the resumed subset; the file holds 40 rows. The manifest keeps attempted 31 for the resumed invocation.
  • Pause to prioritise a drafter test cost 3 hours of wall clock and a mid-suite cut; the drafter test (chad) was then abandoned ("we need to can it, it's not what we need").
  • Identical results looked like a bug: the assistant checked /props (model_ftype PTQ1_0) and that the files differed before accepting it.
  • Stale build stamp / missing flags: as for PQ2_0. Published-data mismatch: the temporal-conflict finding gives bonsai-2 "75%" conflict detection with controls 16/16; recomputed here 24/32 and 16/16, consistent.

What happened to it

Downloaded 2026-09-18 → benched 2026-09-18 → probed → feel-test seat on the Mac Studio 2026-09-19 → M4 Pro seat 2026-09-19 → taken down on the M4 Pro Mac mini by 2026-09-20 → kept on the Mac Studio (registered as the bonsai2 swap-seat candidate in fleetview). Re-downloadable from prism-ml/Ternary-Bonsai-2-27B-gguf.