Model family← Models
Sakura MicroQuality Kolibri-1 365E
A community version of Kolibri-1 with 5% of its experts removed and the rest stored at about 3 bits. Strong with tools, weaker at saying "I don't know", and it now waits in the slot we keep for trying models.
- Files tested
- 2
- Scored runs
- 16
- Smallest Mac
- 48 GB for the file we kept
Made by webmp3
How this page was made. The tests run automatically on our Mac Studio, and every score here comes straight from those runs. This overview was written by Claude, an AI model, from the lab's own records: the run log, notes kept at the time, and the working sessions where each decision was made. Quotes are Alan's words, with spelling fixed. How these pages are made.
Why we tested it
On 11 October we wanted Kolibri-1 in our swap seat, the one slot on our server where models are swapped in to try, but its 44 GB file didn't fit. Alan: "actually, would a smaller quant fit in the swap seat?"
While our agents were planning to make a smaller version themselves, they found this one ready-made. It removes 19 of Kolibri's 384 experts in each layer, about 5%, and stores the rest at about 3 bits. Its maker claims it loses much less than a standard 3-bit file. We wanted to see that on our own tests.
What it took
Like Kolibri-1, it needs llama.cpp built with a community patch. We downloaded a 3-bit and a 2-bit file and tested the 3-bit one.
What happened
- Slower than the full model, but usable. On the same day and build it wrote 51.3 tokens a second, against 70.75 for Kolibri-1's Q4_0: a third smaller and about 28% slower. Alan: "so it's not as fast then but still useful".
- Tools and checking held up. 60 of 60 on our test of when to reach for a tool, 40 of 40 choosing among many tools, 95% at catching errors a first pass let through, and 100% at answering from the document it was given rather than from memory.
- Judgement slipped. On saying "I don't know" it scored 33.3%, against 54.2% for the full Kolibri. That was the biggest gap between them, and with a different setup on each day we can't say whether the pruning, the 3-bit storage or the setup caused it. Turning tool results into an answer it scored 56.7%, and tagging 67.1%.
- So it took no worker's job. Alan made it an option in the swap seat instead: "stick the sakura kolibri as an available model in the swap seat". As he put it, "tbh the swap seat has no real job, it's a play seat, the others are workers".
- Its context grew. Alan found 64K too restrictive, so the seat now gives it two conversations of up to 128K tokens each. Most of our scored tests ran it as four conversations of 32K.
What it taught us
- Pruning and 3-bit kept the tool skills. What dropped most was saying "I don't know". Our two measurements can't separate the causes, so this one stays open.
- A smaller file can be slower. We expected the 3-bit file to run slower than Kolibri's larger 4-bit one, because 3-bit formats take more work per byte, and it did.
- Keep the conversation memory at f16 for Kolibri-family models. With four conversations at once, f16 ran up to 24% faster than 4-bit.
Should you use it?
If you want Kolibri's tool skills in a 29.5 GB file, and can build llama.cpp with the Kolibri patch, it did that job well here. Don't rely on it to say "I don't know", tag text or write up tool results. If you have the memory, the full Kolibri-1 was faster and better at saying "I don't know". For tagging, Gemma 4 12B did far better.
- IQ3_XXS 27.45 GiBkept
The one we tested and kept, 29.5 GB. Available in our swap seat since 11 October.
Webmp3 download; swap-seat bench entry · benched; swap-seat bench entry sakura (live in fleetview) · 29.5 GB · 16 scored runs · smallest Mac 48 GB
- IQ2_XS 19.99 GiB
A smaller 2-bit copy, 21.5 GB, downloaded at the same time. No record of it ever being loaded.
Webmp3 download; never run · On disk · 21.5 GB · never scored
Decisions, in order
- 2026-10-11 12:48Z, assistant surfaces Sakura as a ready-made 2-bit/3-bit pruned candidate.
- 2026-10-11 12:53Z, Alan: "grab the sakura version if it fits, always handy having models to test without fucking around". Both sizes downloaded to
models/kolibri-1-sakura-365e/with adownload.shand sha256 checks. - 2026-10-11 13:42Z, Alan: "do a speed test on sakura" -> seats-down window 13:43-13:48Z. Result: usable, 51 t/s.
- 2026-10-11 13:53Z, Alan: "you can run the gauntlet, i'm going out for a couple hrs" -> unattended gauntlet in the vacated swap seat with a swap watchdog; five other seats stayed up.
- 2026-10-11 17:43-17:47Z, Alan: "so it's not as fast then but still useful"; the swap seat is a play seat; "stick the sakura kolibri as an available model in the swap seat". The assistant opened fleet-roster PR #5 (4 slots x 32K, f16 KV). Alan merged it and restarted fleetview;
sakuraappeared in the bench list at 19:16Z. - 2026-10-12 07:32-07:35Z, Alan asked what context the swap seat serves, answered "yeah np 2" and then "64 still feels quite restrictive". The config went 4 x 32K -> 2 x 64K (fleet-roster commit edb5acc) -> 2 x 128K (commit 607ac14,
-c 262144 -np 2), justified by the Oct 11 needle rerun having run exactly that config for an hour beside the full fleet.
What we learned, in the lab's words
- Speed (same day, same build, seats down): single-stream decode 51.3 t/s vs 70.75 for the unpruned Q4_0; four-stream totals ~85 vs ~104 t/s, i.e. ~21 vs ~26 t/s per stream, still above an edge0 worker's ~15 t/s. The 3-bit pruned file is ~33% smaller and ~28% slower.
- Gauntlet verdict: matches or beats the incumbent on tool work (toolcall 100, toolscale 100, tools 86.7), screening 95, epistemics 100, enrichment 83.3 and contract 75; behind on synthesis (56.7 vs edge0 81.7), grounding (79.2 vs 89.6), tagging (67.1 and 85.0 vs gemma4-12b's ~93 and ~96) and worldview (65 vs gemma 97.5). Not a replacement for any worker seat.
- Unpruned-vs-pruned (two measurements, not one): the Oct 5 Q4_0 ran on build b11396 with one slot; Sakura ran on b11533 with four slots in flight. On the seven shared suites it is similar on toolcall, toolscale, screening, chainwalk and worldview and lower on nullschema and synthesis (table in Sakura Kolibri-1 365E IQ3_XXS 27.45 GiB (webmp3 download; swap-seat bench entry)). The nullschema drop (54.2 -> 33.3) is the first large gap; the cause (pruning vs 3-bit vs the runtime/slot change) is not separable from this data.
- KV cache: f16 beats the house q4_0 KV by 7-10% single-stream and up to 24% at four streams for Kolibri-family models; fleetview's
sakuraentry overrides to f16. - Alan's framing of the swap seat: "tbh the swap seat has no real job, it's a play seat, the others are workers". Gemma carries the only real traffic: "I think it's gemma that's the only hard seat in the whole fleet doing real work".
Why it was tested, in the lab's words
- Alan, 2026-10-11 12:40Z, after the failed first gauntlet window: "actually, would a smaller quant fit in the swap seat?", then "it may be a way of getting the bigger parameter model in the swap as it's the biggest model and the fastest we have".
- The assistant's first plan was to make its own 2-bit mix from a re-downloaded Q8_0, then to build a pruned variant ("REAP-lite"). While checking whether a Kolibri REAP existed it found Sakura: it is "close to what we were about to build" and Alan's own work "becomes the comparison rather than the only option".
- Alan, 12:53Z: "grab the sakura version if it fits, always handy having models to test without fucking around".
- Pre-registered expectations (assistant, written before any Sakura run): the vendor's KL claim "counts for nothing until it walks your benches"; speed "is the thing to measure, not assume" because 3-bit formats cost more compute per byte on the M1; and just before the first number, "slower than the Q4_0's 70.5, as expected".
Open questions
- IQ2_XS has never been loaded or scored (not on record).
- The needle suite: 27 of 110 cases (the longest) were never run at 128K (see Sakura Kolibri-1 365E IQ3_XXS 27.45 GiB (webmp3 download; swap-seat bench entry), Incidents).
- shapegen and sitebuild cannot score any model until the locked suite edit is resolved.
- Whether the nullschema drop vs the full Kolibri is pruning or quantisation: needs the IQ4_XS or a Q4_0 of the same pruned weights, neither held.
- The fleetview
defaultssetreasoning_effort: low, whereas the gauntlet ran theextractionprofile (thinking off); behaviour under real swap-seat use at low effort has no run-log measurement.
The model in detail
| field | value |
|---|---|
| vendor | not Aleph Alpha: the README says "Independent community quantization, not an official Aleph Alpha release" (webmp3, "Sakura Micro" line) |
| upstream repo | webmp3/Sakura-MicroQuality-Kolibri-1-365E-GGUF (three files: IQ2_XS 19.99 GiB, IQ3_XXS 27.45 GiB, IQ4_XS 36.31 GiB); revision and release date not on record |
| base | Aleph-Alpha/Kolibri-1, Apache 2.0 |
| pruning | 19 of 384 routed experts removed per layer (5%), "calibration-selected"; removed experts caught only 0.07% of selections on the vendor's calibration text and 375 of the 950 removed were never selected; calibration covered English, German, code and the model's own reasoning |
| size | 74.4B parameters in the GGUF header against 78.1B unpruned |
| quantisation | importance-matrix mixed IQ quants; vendor KL-divergence claims 50% lower than the standard IQ3_XXS and 58% lower than the standard IQ2_XS at smaller sizes |
| runtime | needs the Hob-forge-patched llama.cpp (llama.cpp-kolibri), not upstream; the README says so and the assistant confirmed the same router gating as the faithful build |
| header trap | general.file_type reads Q8_0: inherited from the Q8_0 the mixer started from; the filename quant is the true one |