MiniCPM5-2B · file← MiniCPM5-2B
EXL3 @4.15 bpw target
An EXL3 copy. Close to the Q8_0 on average across our tests, at about a third of the speed. Deleted. Converted here with PonyExl3.
- Size
- 1.7 GB
- Status
- Deleted 11 Oct 2026
- Scored runs
- 9
- Smallest Mac
- 8 GB estimate, 8K context
| Suite | Score | 95% interval | Passed | Runtime · profile | Recorded |
|---|---|---|---|---|---|
| enrichmentminicpm5-2b-exl3 | 59.3% | 46.0–71.3 | Passed 32/54 | exl3extraction | 2026-09-17 |
| epistemicsminicpm5-2b-exl3 | 81.2% | 68.1–89.8 | Passed 39/48 | exl3study-nothink | 2026-09-17 |
| groundingminicpm5-2b-exl3unenforced80% coverage | 60.0% | 50.2–69.1 | Passed 60/100 | exl3extraction | 2026-09-17 |
| needleminicpm5-2b-exl3 | 85.5% | 77.7–90.8 | Passed 94/110 | exl3extraction | 2026-09-17 |
| screeningminicpm5-2b-exl3 | 93.3% | 84.1–97.4 | Passed 56/60 | exl3extraction | 2026-09-17 |
| synthesisminicpm5-2b-exl3 | 75.0% | 62.8–84.2 | Passed 45/60 | exl3extraction | 2026-09-17 |
| taggerminicpm5-2b-exl3 | 76.2% | 70.5–81.2 | Passed 183/240 | exl3extraction | 2026-09-17 |
| tagger_nullableminicpm5-2b-exl3 | 46.2% | 40.1–52.6 | Passed 111/240 | exl3extraction | 2026-09-17 |
| worldviewminicpm5-2b-exl3 | 37.5% | 24.2–53.0 | Passed 15/40 | exl3extraction | 2026-09-17 |
Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.
Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.
| File | minicpm5-2b-exl3-415/ |
| Format | EXL3 (QTIP trellis), --bits 4.15 target; 4.24 plan average; 4.01 bpw on linears, 5.50 bpw whole file; 1,730,077,136 B |
| Where from | made here 2026-09-16 with ponyexl3-convert from the BF16 safetensors |
| Status | deleted 2026-10-11 |
How it was made
- Input:
models/minicpm5-2b-hf, OpenBMB BF16 safetensors downloaded 2026-09-16 13:41Z (MiniCPM5-2B BF16 safetensors (OpenBMB)). - Tool: PonyExl3 (
github.com/beamivalice/PonyExl3), shallow clone at commit8e7fa6bintobenchmarks/tools/PonyExl3, venv on Python 3.14.7,uv pip install -e ".[dev]". Reported later asponyexl3-0.3.0 mlx-0.31.2 mlx_lm-0.31.3(results manifests). - Command:
.venv/bin/ponyexl3-convert --in-dir models/minicpm5-2b-hf --out-dir models/minicpm5-2b-exl3-415 \
--work-dir ponyexl3-work --bits 4.15 --search-backend metalIt crashed at measure step 60/589 on a macOS GPU watchdog (kIOGPUCommandBufferCallbackErrorImpactingInteractivity) while edge0 and a llama-server shared the GPU. A resume with --buf-size-rows 2048 --max-workers 1 was refused ("work-dir request mismatch"). Everything except Ornith was stopped and it was resumed with the original arguments plus --resume at 17:24Z. Alan had said "You can kill everything other than ornith".
- Completed by 19:29Z:
modules=295 plan_avg_bits=4.237762 layer_reuse=228/824; all linears k=4,lm_headk=6; one HF-standard shard. - Size: 1,730,077,136 B. sha256 not on record.
- What the bit-rate means: quantised linears 4.01 bpw (0.99 GB); token embeddings fp16, unquantised, 0.53 GB (31% of the file); lm_head 6.01 bpw; whole file 5.50 bpw. The "415" in the name is the request, 4.24 is the converter's plan average over modules, and 4.01 is the per-weight figure over linears. The finding's "4.01 bpw" refers to the linears only.
- Who asked: Alan.
How it was served
Served only through benchmarks/tools/ponyexl3_serve.py, a shim written for it (PonyExl3 has no server), launched from the PonyExl3 venv: ponyexl3_serve.py --model models/minicpm5-2b-exl3-415 --name minicpm5-2b-exl3. One slot, ctx 131072, f16 KV per its /props.
| label | dates (UTC) | binary and version | slots / KV / template / reasoning | harness profile | why / status of the batch | ||
|---|---|---|---|---|---|---|---|
minicpm5-2b-exl3 (batch 1) | 2026-09-16 ~20:13Z–21:55Z | shim v1, ponyexl3 8e7fa6b | shim dropped tools; no stop on `< | im_end | >` | extraction, agentic-fast | first gauntlet; toolcall/toolscale 0% were the shim dropping tools ("those are my bug, not the model's") — retired to results/_retired/, not in the run log |
minicpm5-2b-exl3 (batch 2) | 2026-09-16 21:56Z–22:32Z | shim v2 (refuses tools) | still no stop token, no reasoning split, sampler dropped | extraction | 8 suites queued; worldview (40) and synthesis (13 of 60) landed before the run was killed for the DeepSeek/Colibrì test. Retired 2026-09-17 14:20Z (results/retired/README-exl3-2026-09-17.md), but both rows remain in the run log | ||
minicpm5-2b-exl3 (batch 3) | 2026-09-17 14:20Z–15:04Z | shim v3 (stop ids [1,130073], reasoning split, sampler wired) | chat_template_kwargs dropped, so thinking ON | extraction, study-nothink | epistemics, screening, tagger (29 rows) — retired (results/retired/README-exl3-thinking-confound.md): "a comparison between two different configurations of the model"; not in the run log | ||
minicpm5-2b-exl3 (batch 4) | 2026-09-17 15:05Z–16:39Z | shim v4 (+ chat_template_kwargs forwarded) | thinking off, 1 slot | extraction, study-nothink | the gauntlet the finding uses: 9 suites (shapegen refused: dataset-lock drift) |
The run-log llama_build field on every row is 0.4.0-dev (build 10893, commit c32d1dabe), the llama.cpp build, which did not serve any of them (stale stamp; runtime / runtime_version are correct).
Decisions
- Built as the first EXL3 test on Apple Silicon. Kept through the 2026-09-16 cleanup, "flagged untested because I converted it forty minutes ago".
- Three batches retired, not deleted, each with a note.
- Verdict: "for a 2B model, 35% less memory doesn't pay unless memory is what's binding". Never seated.
- On the 2026-10-07 publish review: "matches Q8_0 on capability, 3x slower, needs an EXL3 runtime; licence not recorded". Not published.
What went wrong
- Four shim defects, each producing a plausible wrong number: tools dropped (0% tool suites); no stop on
<|im_end|>(runaway to max_tokens, "correct answer first, then ~100 repetitions"); reasoning glued tocontent; sampler andchat_template_kwargsdropped (both READMEs inresults/retired/). - Mis-stated bit-rate: the directory name says 4.15; the true whole-file figure is 5.50 bpw. The shim now reports both.
- Conversion crash on the GPU watchdog under contention (section 1).
- Stale build stamp:
llama_buildnames the llama.cpp build on all 11 rows.
What happened to it
Converted 2026-09-16 → batches 1–3 retired → benched 2026-09-17 (batch 4) → never seated → deleted 2026-10-11 (the deletion notes call it never benched, which is wrong). Not on HF. Recreate: download openbmb/MiniCPM5-2B and rerun the command above with PonyExl3 8e7fa6b (~2 h on this box with the GPU otherwise idle).