Ternary-Bonsai-27B · file← Ternary-Bonsai-27B
MLX 2-bit
PrismML's MLX copy, tested on two Macs: within a few points of the GGUF's scores, and faster. Deleted from the Mac Studio on 11 October. Prism-ml `-mlx-2bit`.
- Size
- 8.5 GB
- Status
- Deleted 11 Oct 2026
- Scored runs
- 26
- Smallest Mac
- 16 GB estimate, 8K context
Results26 scored · 95% intervals
| Suite | Score | 95% interval | Passed | Runtime · profile | Recorded |
|---|---|---|---|---|---|
| chainwalkbonsai-27b-mlx2-m4pro | 0.0% | 0.0–65.8 | Passed 0/2 | mlx_lmagentic-fast | 2026-09-14 |
| contractbonsai-27b-mlx2-cortext | 75.0% | 61.2–85.1 | Passed 36/48 | mlx_lmextraction | 2026-09-14 |
| contractbonsai-27b-mlx2-m4pro | 75.0% | 61.2–85.1 | Passed 36/48 | unknownextraction | 2026-09-14 |
| enrichmentbonsai-27b-mlx2-cortext | 72.2% | 59.1–82.4 | Passed 39/54 | mlx_lmextraction | 2026-09-15 |
| enrichmentbonsai-27b-mlx2-m4pro | 72.2% | 59.1–82.4 | Passed 39/54 | unknownextraction | 2026-09-14 |
| epistemicsbonsai-27b-mlx2-cortext | 89.6% | 77.8–95.5 | Passed 43/48 | mlx_lmstudy-nothink | 2026-09-14 |
| epistemicsbonsai-27b-mlx2-m4pro | 89.6% | 77.8–95.5 | Passed 43/48 | mlx_lmstudy-nothink | 2026-09-14 |
| groundingbonsai-27b-mlx2-cortextunenforced | 71.2% | 62.7–78.4 | Passed 89/125 | mlx_lmextraction | 2026-09-14 |
| groundingbonsai-27b-mlx2-m4prounenforced | 71.2% | 62.7–78.4 | Passed 89/125 | mlx_lmextraction | 2026-09-14 |
| screeningbonsai-27b-mlx2-cortext | 93.3% | 84.1–97.4 | Passed 56/60 | mlx_lmextraction | 2026-09-14 |
| screeningbonsai-27b-mlx2-m4pro | 93.3% | 84.1–97.4 | Passed 56/60 | mlx_lmextraction | 2026-09-14 |
| screeningternary-bonsai-27b-recheck | 93.3% | 84.1–97.4 | Passed 56/60 | mlx_lmextraction | 2026-09-17 |
| synthesisbonsai-27b-mlx2-cortext | 66.7% | 54.1–77.3 | Passed 40/60 | mlx_lmextraction | 2026-09-14 |
| synthesisbonsai-27b-mlx2-m4pro | 66.7% | 54.1–77.3 | Passed 40/60 | mlx_lmextraction | 2026-09-14 |
| taggerbonsai-27b-mlx2-cortext | 87.5% | 82.7–91.1 | Passed 210/240 | mlx_lmextraction | 2026-09-15 |
| taggerbonsai-27b-mlx2-m4pro | 87.9% | 83.2–91.5 | Passed 211/240 | unknownextraction | 2026-09-14 |
| tagger_nullablebonsai-27b-mlx2-cortext | 88.8% | 84.1–92.2 | Passed 213/240 | mlx_lmextraction | 2026-09-15 |
| tagger_nullablebonsai-27b-mlx2-m4pro | 89.2% | 84.6–92.5 | Passed 214/240 | unknownextraction | 2026-09-14 |
| toolcallbonsai-27b-mlx2-cortext | 100.0% | 94.0–100.0 | Passed 60/60 | mlx_lmextraction | 2026-09-14 |
| toolcallbonsai-27b-mlx2-m4pro | 100.0% | 94.0–100.0 | Passed 60/60 | mlx_lmextraction | 2026-09-14 |
| toolsbonsai-27b-mlx2-cortext | 100.0% | 79.6–100.0 | Passed 15/15 | mlx_lmagentic-fast | 2026-09-14 |
| toolsbonsai-27b-mlx2-m4pro | 100.0% | 79.6–100.0 | Passed 15/15 | mlx_lmagentic-fast | 2026-09-14 |
| toolscalebonsai-27b-mlx2-cortext | 100.0% | 91.2–100.0 | Passed 40/40 | mlx_lmagentic-fast | 2026-09-14 |
| toolscalebonsai-27b-mlx2-m4pro | 100.0% | 91.2–100.0 | Passed 40/40 | mlx_lmagentic-fast | 2026-09-14 |
| worldviewbonsai-27b-mlx2-cortext | 42.5% | 28.5–57.8 | Passed 17/40 | mlx_lmextraction | 2026-09-14 |
| worldviewbonsai-27b-mlx2-m4pro | 42.5% | 28.5–57.8 | Passed 17/40 | mlx_lmextraction | 2026-09-14 |
Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.
The recordHow it was made · how it was served · what went wrong · what happened to it
Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.
| File | model.safetensors, 8,490,785,104 B (two copies: the Mac Studio, M4 Pro) |
| Format | MLX 2-bit g128 |
| Where from | downloaded 2026-09-14 |
| Status | the Mac Studio copy deleted 2026-10-11; M4 Pro copy not on record |
| Hugging Face | https://huggingface.co/prism-ml/Ternary-Bonsai-27B-mlx-2bit |
Where it came from
- Copy 1, the Mac Studio:
models/ternary-bonsai-27b-mlx2/,snapshot_downloadon 2026-09-14, started 16:38Z, complete by 16:45Z. The revision pinned is not on record for this copy; HFmainwas unchanged since 2026-07-14, so it is most likely 70f75f3 (reconstructed). A byte comparison with copy 2 was not made. - Copy 2, the M4 Pro Mac mini: the HF cache snapshot above, put there by Alan before 16:54Z ("it's already on the m4 pro :)"). Used for the
bonsai-27b-mlx2-m4proruns. The copy's later fate is not on record. - Runtime: stock
mlx_lmserver. The card says prism's MLX fork is required for the custom kernels; the assistant tested stock first, since the card's own install line ispip install mlx-lm. The M4 Pro's reply to a sanity check, 7.7 GB resident, showed "real 2-bit kernels, no fp16 expansion". mlx_lm version: not on record for either gauntlet (the Mac Studio useduv run --with mlx-lm, latest at the time);0.31.3for the 2026-09-17 recheck (run-log). - Earlier copy: a 7.9G
ternary-bonsai-27b-mlxdirectory existed from at least 2026-08-23 and was deleted 2026-09-07 (Ternary-Bonsai-27B MLX 2-bit, first copy (deleted 2026-09-07)).
How it was served
| label | dates (UTC) | binary and version | ctx / -np / KV / -fa / --jinja / --reasoning-format | harness profile | why this label exists |
|---|---|---|---|---|---|
bonsai-27b-mlx2-m4pro | 2026-09-14 16:59Z launch, run-log 18:04:25 to 23:33:48 BST; eight suites plus chainwalk (the assistant counted nine) until the server died (found 20:16Z), then four suites re-run from 20:39Z | mlx_lm server started by Alan on the M4 Pro; version not on record. The last four run-log rows are unknown because the server was restarted by Alan without the command being recorded (run-log) | n/a (mlx_lm: no --jinja; chat_template_kwargs.enable_thinking=false per request, honoured, zero reasoning chars); model id = the filesystem snapshot path | extraction / agentic-fast / study-nothink | Alan: "nah go for it whilst it's up, m4 pro can probably handle a seat at some point so running benchmarks there also a good idea". Cross-machine point for the speed question. |
bonsai-27b-mlx2-cortext | server 19:58Z, gauntlet launched 20:18Z; run-log 21:22:56 BST to 03:10:44 BST (2026-09-15) | uv run --quiet --with mlx-lm python -m mlx_lm server --model models/ternary-bonsai-27b-mlx2; version not on record | n/a | same harness profiles | Re-run on the Mac Studio because the M4 Pro run varied machine and runtime together: "supersedes it as evidence" (assistant). The chainwalk result file has 0 rows. |
ternary-bonsai-27b-recheck | 2026-09-17 17:41Z; screening only | benchmarks/tools/PonyExl3/.venv python, mlx_lm 0.31.3 (run-log) | n/a | extraction | Provenance spot-check of the board after a baseline would not reproduce: recorded 93.3, re-measured 93.3. |
| (no label) speed legs | the Mac Studio 2026-09-14 19:59Z; M4 Pro 16:57Z and 22:56Z | as above | n/a | n/a | The cross-machine decode comparison (below). |
Seats held: none. The Mac Studio server was stopped by the overnight script after the gauntlets cleared at about 04:10 BST on 2026-09-15.
Decisions
- Why downloaded: Alan asked "the bonsai one has 2? native mlx and normal" and the assistant found the MLX pack was the build prism publishes Apple-Silicon numbers for. After the GGUF showed 17.4 tok/s against prism's 26.2 on an M5 Pro, the assistant could not tell machine from runtime, and Alan: "yes, we have to, this is an important seat". Alan: "I can stand up the same model on my m4 pro to test this too".
- Alan, on contention: "the model on [the Mac Studio] may suffer for benchmarks running too for both boxes". The assistant checked CPU, memory, slots and logs and kept the concurrent runs.
- Alan, 2026-09-14 21:20Z, after the MLX/GGUF pairing: "So bonsai is definitely a good idea then".
What went wrong
- M4 Pro server died mid-gauntlet; the script still printed COMPLETE.
contract,enrichment,taggerandtagger_nullablegotConnection refusedfor 582 cases; the harness correctly reported coverage 0 instead of zeros. Alan: "try them again". The assistant first pivoted to the Mac Studio MLX run, then admitted it had not retried the M4 Pro suites; the four result files went to_retiredand were re-run from 20:39Z. Cost: about 90 minutes and a runtime field ofunknownon four run-log rows. - Chainwalk gap: the
bonsai-27b-mlx2-m4prochainwalk (n=2) scores 0 of 2 and the Mac Studio chainwalk file has 0 rows (manifest says 2 attempted). The "-62 point" GGUF vs MLX chainwalk gap quoted in chat was n=2 noise. - Stale build stamp (standing check): the run-log
llama_buildforternary-bonsai-27b-recheckis the stock llama.cpp build though the run was MLX (runtime recordedmlx_lm 0.31.3). - Spot-check comparability: the recheck reproduced 93.3 on MLX weights through mlx_lm, while the number it was checked against was the GGUF PQ2_0 screening run. Equal numbers across two runtimes are a good sign, but this is not a same-runtime reproduction.
- Missing flags: not applicable to mlx_lm; no chat-template kwargs were set on the server (per-request).
- Partial runs: the Mac Studio chainwalk file (0 rows of 2) as above; all other files are complete.
What happened to it
Cortext copy: downloaded 2026-09-14 → benched on both machines 2026-09-14/15 → recheck 2026-09-17 → deleted 2026-10-11 (directory ternary-bonsai-27b-mlx2, 7.9G, in the second sweep; RESTORE-deleted-20261011-tiersAB.sh calls it "never benched (mlx2 lineage)", which is wrong: 25 run-log rows were scored on this weights file). Re-downloadable from prism-ml/Ternary-Bonsai-27B-mlx-2bit. The HF hub cache on the Mac Studio still holds a 4 KB shell for the repo (read 2026-10-23). The M4 Pro copy: not on record.