FreshContextA context lab · Fife
Drawing
02 · Benchmarks
Sheet
2 of 3
Rev
A · 2026-10-01

Ternary-Bonsai-2-27B · file← Ternary-Bonsai-2-27B

Ternary-Bonsai-2-27B PQ2_0

PrismML's 2.13-bit file, and the first we tested. Same answers as the smaller file; still on disk. Prism-ml, prism fork.

Size
7.2 GB
Status
On disk
Scored runs
8
Smallest Mac
16 GB estimate, 8K context
Results8 scored · 95% intervals
SuiteScore95% interval PassedRuntime · profileRecorded
epistemicsbonsai2-27b-pq2100.0%92.6–100.0Passed 48/48llama.cppstudy-nothink2026-09-18
groundingbonsai2-27b-pq266.4%57.7–74.1Passed 83/125llama.cppextraction2026-09-18
nullschemabonsai2-27b-pq231.9%22.3–43.4Passed 23/72llama.cppextraction2026-09-18
screeningbonsai2-27b-pq298.3%91.1–99.7Passed 59/60llama.cppextraction2026-09-18
synthesisbonsai2-27b-pq283.3%72.0–90.7Passed 50/60llama.cppextraction2026-09-18
toolcallbonsai2-27b-pq2100.0%94.0–100.0Passed 60/60llama.cppextraction2026-09-18
toolscalebonsai2-27b-pq297.5%87.1–99.6Passed 39/40llama.cppagentic-fast2026-09-18
worldviewbonsai2-27b-pq282.5%68.0–91.3Passed 33/40llama.cppextraction2026-09-18

Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.

Corrections8 to the record published 21 October 2026

  • Build of llama.cpp on 8 runs was 0.4.0-dev (build 10893, commit c32d1dabe), now 0.2.0-dev (build 1, commit 6ac5eb0). Served by the PrismML llama.cpp fork (build 1, commit 6ac5eb0), as every run manifest records; the stamp named the stock binary instead.

    Corrected 24 October 2026 · a mechanical fix

The recordHow it was made · how it was served · what went wrong · what happened to it

Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.

FileTernary-Bonsai-2-27B-PQ2_0.gguf, 7,206,168,928 B
FormatGGUF 2.13 bpw
Where fromdownloaded 2026-09-18
Statuson disk (the Mac Studio), also on the M4 Pro
Hugging Facehttps://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf

Where it came from

  • Source: prism-ml/Ternary-Bonsai-2-27B-gguf, file Ternary-Bonsai-2-27B-PQ2_0.gguf. The repo listed F16 (53.81 GB), PQ2_0 (7.21 GB), PTQ1_0 (5.95 GB) and two mmproj files on 2026-09-18.
  • Downloaded: 2026-09-18 06:25Z, complete by 06:29Z, hf_hub_download with HF_XET_HIGH_PERFORMANCE=1 from the PonyExl3 venv python into models/bonsai2-27b/. Revision and sha256: not on record.
  • Header facts: /props reports model_ftype PQ2_0 - 2.13 bpw (group 128) in every manifest; the chat-template hash stamped is c3cf9e34abf4f9e3 (manifests).
  • Runtime: the PrismML fork built on 2026-09-14 (llama.cpp-prism, commit 6ac5eb0, 0.2.0-dev (build 1)), which the manifests record as b1-6ac5eb0. Upstream cannot load the type. A second prism build, prism-b10709-9a9394a (0.2.0-dev, build 10709), was installed on the M4 Pro (a lab note).

How it was served

labeldates (UTC)binary and versionctx / -np / KV / -fa / --jinja / --reasoning-formatharness profilewhy this label exists
bonsai2-27b-pq22026-09-18 06:29Z start; run-log 07:32:38 to 08:11:14 BST (06:32Z to 07:11Z); 8 of 14 suitesllama.cpp-prism/build/bin/llama-server, b1-6ac5eb0 (manifests and run-log runtime_version 6ac5eb0; run-log llama_build names stock b10893, stale)16384 total / 1 / f16 (no KV flags) / default / on / not on the line (-ngl 99 --no-warmup --alias bonsai2-27b)extraction, agentic-fast (toolscale), study-nothink (epistemics)The eight suites that pair with existing Qwen3.8-27B baselines. Alan: "Oh snap, bonsai 2 did qwen 3.8 ternary".
(no label) speed probe2026-09-18 07:19Z to 07:20Zsame binary8192 / 1 / f16 / default / on / not on the line (--alias bonsai2)n/aClean decode on the shape of the vendor benchmark: 512-token prompt, 128-token greedy decode.
(no label) draft-model attempt07:22Z to 07:34Zsame binary-md Bonsai-2-27B-DFlash2-Q8_0.gguf -ngl 99 -ngld 99 -c 8192 -np 1n/aTest the vendor-adjacent 3x drafter claim on Metal; failed to load (see the drafter's page).

Seats held: none from this file on the Mac Studio. On the M4 Pro the PQ2_0 file "is also on disk" next to the seat's PTQ1_0 (a lab note); that copy's provenance is not on record.

Decisions

  • Why downloaded: Alan, 2026-09-18 06:23Z: "Oh snap, bonsai 2 did qwen 3.8 ternary". The assistant's framing: this is the case Alan's own finding predicted would work (a natively ternary-trained model in a ternary container) and Qwen3.8-27B baselines exist. Alan had asked the speed question first: "From memory was the ternary version faster or slower due to the unpack?"; the recorded answer was that Bonsai-1 ternary and Qwen3.8 Q4_0 both median 7.6 to 7.8 tok/s, half the bytes at the same speed.
  • Result (assistant, 07:11Z): parity with its own Q4_0 on synthesis, toolcall and worldview, a 37.5-point nullschema loss, and a mean of -7.3 over the paired suites. The finding was published at 07:11Z.
  • Alan's reading after the PTQ1_0 identity result: "man that's kind of depressing but at least the seat is smaller". The assistant's summary: every route to more speed landed on the same number; the win is footprint.
  • PQ2_0 vs PTQ1_0: identical output on eight suites, 505 of 505 responses byte-identical, so the smaller PTQ1_0 file is the deployment file (PTQ1_0 and PQ2_0 produce byte-identical output on a ternary-trained model, so the smaller file is free; Ternary-Bonsai-2-27B PTQ1_0 (prism-ml, 1.75 bpw)). PQ2_0 stayed on disk beside it.

What went wrong

  • False VOID warning (resolved): the harness warned that every toolcall response was empty and the run should be treated as void; the assistant inspected the rows (outcome: called) and found the Q4_0 baseline has the identical shape; the 100.0 stands.
  • Mixed baselines in the headline comparison (published-data mismatch): the finding said "against its own Q4_0" and "-7.3 over seven paired suites". Five of the seven pairings are against the Q4_0 label; epistemics (100.0) and grounding (80.0, the -13.6 loss) are against the UD-Q4_K_XL runs of 2026-08-24 on build b10369. The comparison is a mix of two Qwen3.8 builds on two llama.cpp builds. The conclusion (nullschema falls most) is unaffected: nullschema is a Q4_0-label pairing.
  • Partial runs (standing check): 8 suites only; every result file holds as many rows as its suite has cases. Stale build stamp: run-log llama_build names upstream b10893 while runtime_version and the manifests name the prism fork. Missing flags: --jinja on; no --reasoning-format on the line; no KV flags (f16 KV, one slot).
  • Naming trap: Alan calls the PTQ1_0 file "the 1bit" ("What that on the 1bit though?", "in theory if I drop ornith, I can have edge AND 1bit bonsai 2?"). PTQ1_0 is a 1.75 bpw ternary container, not the Q1_0 1-bit Bonsai family (family-bonsai-27b-1bit). His later "is it the 1bit version?" has no recorded answer.

What happened to it

Downloaded 2026-09-18 → benched 2026-09-18 (8 suites) → kept as the second file of models/bonsai2-27b/ (12 GB with the PTQ1_0) → still on disk at the 2026-10-23 read (ls shows both files dated 18 Sep). Not touched by the 2026-10-11 sweep (the sweep note keeps "bonsai2-27b PTQ1_0 (kept, 12G)", RESTORE-deleted-20261011-tiersAB.sh). Re-downloadable from prism-ml/Ternary-Bonsai-2-27B-gguf.