- Parameters
- 12B
- Quant
- q4_0 QAT
- Size
- 7.0 GB
- Family
- gemma4
gemma4-12b
Seated · Tagdenseresidentllama.cpp
Model card ↗Outcome as of 20 September 2026, from the seat registry. The write-up below was made when the model was tested.
Why it was tested
Tested as a cheap tagger after the 80B proved expensive for a task that runs constantly.
What came of it
Holds the tagging seat at 93.8/97.9, beating the 80B by 7.5 points. On the extraction suites it is statistically indistinguishable from Qwen3-Next-80B at one sixth the size, which is the most uncomfortable result on this board.
| Suite | Score | 95% interval | Passed | Runtime · profile | Recorded |
|---|---|---|---|---|---|
| chainwalk | 50.0% | 9.5–90.5 | Passed 1/2 | llama.cppagentic-fast | 2026-08-24 |
| contract | 66.7% | 50.3–79.8 | Passed 24/36 | llama.cppextraction | 2026-09-08 |
| enrichment | 77.8% | 65.1–86.8 | Passed 42/54 | llama.cppstudy-nothink | 2026-08-24 |
| epistemics | 100.0% | 92.6–100.0 | Passed 48/48 | llama.cppstudy-nothink | 2026-08-24 |
| grounding | 84.0% | 76.6–89.4 | Passed 105/125 | llama.cppextraction | 2026-08-24 |
| nullschema | 88.9% | 79.6–94.3 | Passed 64/72 | llama.cppextraction | 2026-08-24 |
| screening | 93.3% | 84.1–97.4 | Passed 56/60 | llama.cppextraction | 2026-08-24 |
| synthesis | 83.3% | 72.0–90.7 | Passed 50/60 | llama.cppextraction | 2026-09-09 |
| tagger | 75.0% | 69.2–80.1 | Passed 180/240 | llama.cppstudy-nothink | — |
| tools | 86.7% | 62.1–96.3 | Passed 13/15 | llama.cppagentic-fast | 2026-08-24 |
| toolscale | 90.0% | 76.9–96.0 | Passed 36/40 | llama.cppagentic-fast | 2026-08-24 |
| worldview | 97.5% | 87.1–99.6 | Passed 39/40 | llama.cppextraction | 2026-09-09 |
Runs marked unenforced were measured through mlx_lm on a suite that requires constrained decoding. mlx_lm accepts the schema and ignores it, so those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.
Every run log entry for this model (13)
| Finished | Suite | Runtime · profile | Score | Coverage | Grader · dataset · config |
|---|---|---|---|---|---|
| worldviewworldview/2 | not recordedextraction | 97.5% | Coverage 40/40 | g 9acfba1b704feb28d e97835de9e6fc227c 585e7383f726d519 | |
| synthesissynthesis/2 | not recordedextraction | 83.3% | Coverage 60/60 | g 4156dd3c13361e8bd 267d1954e225c5b6c dea865f1d077f0d8 | |
| synthesissynthesis/1 | not recordedextraction | 80.0% | Coverage 60/60 | g 27df090db4445c2cd 760936a5348b2ffdc 9749137aeec83ad6 | |
| contractcontract/1 | not recordedextraction | 66.7% | Coverage 36/36 | g 2dd146852463f032d ab4bb33923bd0ed6c 0d1de531e8b00c72 | |
| screeningscreening/1 | llama.cppextraction | 93.3% | Coverage 60/60 | g 0661b49a10be2d2ad 64c97ca2269c29afc b7d605769821c0d8 | |
| nullschemanullschema/1 | llama.cppextraction | 88.9% | Coverage 72/72 | g d7448e0f5fc3f9afd 3b91280bb40372efc ec3358e6aa5e1088 | |
| groundinggrounding/1 | llama.cppextraction | 84.0% | Coverage 125/125 | g 30b933dcc6a7e2d4d dcb396faf8b83b0ac 31241ed46ed4d6a5 | |
| toolscaletoolscale/1 | llama.cppagentic-fast | 90.0% | Coverage 40/40 | g 1bae1f3e1b606eaed b24abb108c409178c 42412bb32a456ced | |
| enrichmentenrichment/3 | llama.cppstudy-nothink | 77.8% | Coverage 54/54 | g 7d661b93852b5ffdd 75361864baa727abc 5367be2a63165de3 | |
| epistemicsepistemics/4 | llama.cppstudy-nothink | 100.0% | Coverage 48/48 | g 2a2f7e4e90d6dd63d 4bce453fd376553cc b6ac24eba6a83a5d | |
| chainwalkchainwalk/2 | llama.cppagentic-fast | 50.0% | Coverage 2/2 | g 48abd22e0117fddad 129a9e74408c8919c b4834ee71de7e834 | |
| toolstools/1 | llama.cppagentic-fast | 86.7% | Coverage 15/15 | g d6a32c09e57bea7bd 33a44812f0ad4c39c b34ee497223231c8 | |
| taggertagger/2 | llama.cppstudy-nothink | 93.2% | Coverage 80.3% coverage | g 28983856e4bf2dd9d 7ef587bf33e5788ac d10e58be49cb1d55 |
- Gemma 4 E2BMeasured, not seated
2B · dense · q4_0 · 5.0 GB
The base model for the tagger fine-tune, and the smallest thing worth measuring.
- tagger-e2b-v1 (LoRA fine-tune)Rejected
2B · dense · MLX 4-bit · 3.0 GB
A custom tagger, fine-tuned with MLX LoRA on 270 turns, intended to replace the general model in the tagging seat.
- 2026-09-20Typed decisions split across a routing table, not a winner; the only arm you can put a threshold on matters more than the most accurate one
- 2026-09-17One baseline was 25 points off and nothing in the stamp could explain it; the other six reproduced exactly
- 2026-09-10Detecting emptiness and rejecting a plausible candidate are different capabilities
- 2026-09-10gemma4-12b does not act on rules placed in tool descriptions
- 2026-09-10The 6.7 GB tagger is the most epistemically reliable model in the fleet
- 2026-09-09No model qualifies single-source evidence, even when explicitly told to
- 2026-09-08The extraction suites can no longer separate the leading models
- 2026-09-07The tagger fine-tune scored below its own base model
- 2026-09-07Specialist labels failed to predict outcomes three times in four