Suite · tagger_nullable← Suites
Tagger with absence permitted
Does allowing empty fields change tagging behaviour?
- Cases
- 240
- Scored runs
- 24
- Models
- 24
- Top score
- 97.9%
SpecificationMethod · grading · caveat
- Method
- The same 240 turn pairs and the same grader as the tagger suite, with the schema permitting empty topic and entity lists.
- Grading
- Identical to the tagger suite.
- Caveat
- Paired with tagger by design. The two should be read together, never separately.
- Version
tagger-nullable/1- Cases
- 240
Recorded runs24 runs · sorted by accuracy
Sorted by accuracy. The bar shows the 95 percent confidence interval. Where intervals overlap, the models are not distinguishable on this suite at this number of cases.
Top of the suite gemma4-12b-ab 97.9% 95% interval 95.2–99.1
| Model | Score | 95% interval | Passed | Runtime · profile | Recorded |
|---|---|---|---|---|---|
| gemma4-12b-ab | 97.9% | 95.2–99.1 | Passed 235/240 | llama.cppextraction | 2026-08-24 |
| gemma4-12b-b10618 | 97.9% | 95.2–99.1overlaps the top interval | Passed 235/240 | llama.cppextraction | 2026-08-31 |
| qwen3.5-4b-base | 93.8% | 89.9–96.2overlaps the top interval | Passed 225/240 | llama.cppextraction | 2026-09-13 |
| qwen3.6-35b-tag | 92.5% | 88.5–95.2overlaps the top interval | Passed 222/240 | llama.cppextraction | 2026-08-31 |
| BTL-4 Compact (IQ2_XXS)btl-4 | 91.7% | 87.5–94.5 | Passed 220/240 | llama.cppextraction | 2026-09-15 |
| edge0-35b | 91.7% | 87.5–94.5 | Passed 220/240 | llama.cppextraction | 2026-09-16 |
| Bonsai-27B (Q1_0)bonsai-27b-1bit | 90.8% | 86.5–93.9 | Passed 218/240 | llama.cppextraction | 2026-09-15 |
| gemma4-e2b-ab | 90.4% | 86.0–93.5 | Passed 217/240 | llama.cppextraction | 2026-08-24 |
| qwen3-next-80b-tag | 90.4% | 86.0–93.5 | Passed 217/240 | llama.cppextraction | 2026-08-31 |
| Qwen3.8-9B-Distill (Q8_0)qwen3.8-9b-distill | 90.0% | 85.6–93.2 | Passed 216/240 | llama.cppextraction | 2026-09-14 |
| Ternary-Bonsai-27B (prism-ml)ternary-bonsai-27b | 90.0% | 85.6–93.2 | Passed 216/240 | llama.cppextraction | 2026-09-14 |
| bonsai-27b-mlx2-m4pro | 89.2% | 84.6–92.5 | Passed 214/240 | mlxextraction | 2026-09-14 |
| bonsai-27b-mlx2-cortext | 88.8% | 84.1–92.2 | Passed 213/240 | mlxextraction | 2026-09-15 |
| gpt-oss-20b-study-default | 88.6% | 74.0–95.5overlaps the top interval | Passed 31/35 | llama.cppstudy-default | — |
| neohorse-1-4b | 86.2% | 81.3–90.0 | Passed 207/240 | llama.cppextraction | 2026-09-13 |
| gpt-oss-20b | 85.0% | 79.9–89.0 | Passed 204/240 | llama.cppextraction | — |
| tagger-e2b-v1-dl-tagger-e2b-v1 | 75.0% | 69.2–80.1 | Passed 180/240 | mlxextraction | 2026-09-07 |
| gemma4-e2b-mlx-4bit | 73.8% | 67.8–78.9 | Passed 177/240 | mlxextraction | 2026-09-07 |
| ternary-bonsai-8b | 61.2% | 55.0–67.2 | Passed 147/240 | llama.cppextraction | 2026-09-17 |
| minicpm5-2b-q8-np1 | 52.9% | 46.6–59.1 | Passed 127/240 | llama.cppextraction | 2026-09-17 |
| BTL-4 (Q4_K_M)btl-4-q4km | 48.8% | 42.5–55.0 | Passed 117/240 | llama.cppextraction | — |
| minicpm5-2b-exl3 | 46.2% | 40.1–52.6 | Passed 111/240 | llama.cppextraction | 2026-09-17 |
| Bonsai-8B (Q1_0)bonsai-8b-1bit | 41.7% | 35.6–48.0 | Passed 100/240 | llama.cppextraction | 2026-09-15 |
| minicpm5-2b | 22.9% | 18.1–28.6 | Passed 55/240 | llama.cppextraction | 2026-09-15 |
Run log25 logged runs of this suite
Every logged run, newest first (25)
| Finished | Model | Runtime · profile | Score | Coverage | Grader · dataset · config |
|---|---|---|---|---|---|
| minicpm5-2b-q8-np1 | not recordedextraction | 52.9% | Coverage 240/240 | g 04e875431e26c4b5d db3addf23a2bc4cec 070be56608b35405 | |
| minicpm5-2b-exl3 | not recordedextraction | 46.2% | Coverage 240/240 | g 04e875431e26c4b5d db3addf23a2bc4cec dfef0a54e96403a1 | |
| ternary-bonsai-8b | not recordedextraction | 61.2% | Coverage 240/240 | g 04e875431e26c4b5d db3addf23a2bc4cec 1a6187eaf0f19f79 | |
| edge0-35b | not recordedextraction | 91.7% | Coverage 240/240 | g 04e875431e26c4b5d db3addf23a2bc4cec 7b18c32f56523945 | |
| minicpm5-2b | not recordedextraction | 22.9% | Coverage 240/240 | g 04e875431e26c4b5d db3addf23a2bc4cec 8d6eef850a9bedcc | |
| btl-4-q4km | not recordedextraction | 71.3% | Coverage 68.3% coverage | g 04e875431e26c4b5d db3addf23a2bc4cec 53f41ca547c1a228 | |
| btl-4-q4km | not recordedextraction | 71.3% | Coverage 164/164 | g 04e875431e26c4b5d db3addf23a2bc4cec 53f41ca547c1a228 | |
| btl-4 | not recordedextraction | 91.7% | Coverage 240/240 | g 04e875431e26c4b5d db3addf23a2bc4cec f9c64867e9dfb91b | |
| bonsai-27b-1bit | not recordedextraction | 90.8% | Coverage 240/240 | g 04e875431e26c4b5d db3addf23a2bc4cec f3c4a61093079a3e | |
| bonsai-27b-mlx2-cortext | mlxextraction | 88.8% | Coverage 240/240 | g 04e875431e26c4b5d db3addf23a2bc4cec 67cfcfc5c61806c7 | |
| bonsai-8b-1bit | not recordedextraction | 41.7% | Coverage 240/240 | g 04e875431e26c4b5d db3addf23a2bc4cec d5d5bc2627ca6b54 | |
| bonsai-27b-mlx2-m4pro | mlxextraction | 89.2% | Coverage 240/240 | g 04e875431e26c4b5d db3addf23a2bc4cec 29c36e514c29b1c2 | |
| ternary-bonsai-27b | not recordedextraction | 90.0% | Coverage 240/240 | g 04e875431e26c4b5d db3addf23a2bc4cec 29960d736fce3ea5 | |
| qwen3.8-9b-distill | not recordedextraction | 90.0% | Coverage 240/240 | g 04e875431e26c4b5d db3addf23a2bc4cec d7923863e8b56e76 | |
| qwen3.5-4b-base | not recordedextraction | 93.8% | Coverage 240/240 | g 04e875431e26c4b5d db3addf23a2bc4cec 50a352dbe249d795 | |
| neohorse-1-4b | not recordedextraction | 86.2% | Coverage 240/240 | g 04e875431e26c4b5d db3addf23a2bc4cec 78196673fb9772d8 | |
| gpt-oss-20b-study-default | not recordedstudy-default | 93.9% | Coverage 94.3% coverage | g 04e875431e26c4b5d db3addf23a2bc4cec 7649f91ca5eb87a4 | |
| gpt-oss-20b | not recordedextraction | 85.7% | Coverage 99.2% coverage | g 04e875431e26c4b5d db3addf23a2bc4cec acb0e0e0f1883336 | |
| tagger-e2b-v1-dl-tagger-e2b-v1 | llama.cppextraction | 75.0% | Coverage 240/240 | g 04e875431e26c4b5d db3addf23a2bc4cec b0015c0679b3c9f8 | |
| gemma4-e2b-mlx-4bit | mlxextraction | 73.8% | Coverage 240/240 | g 04e875431e26c4b5d db3addf23a2bc4cec d40c89a4c0ded16e | |
| qwen3-next-80b-tag | llama.cppextraction | 90.4% | Coverage 240/240 | g 04e875431e26c4b5d db3addf23a2bc4cec c033312a94ce8bbe | |
| qwen3.6-35b-tag | llama.cppextraction | 92.5% | Coverage 240/240 | g 04e875431e26c4b5d db3addf23a2bc4cec 2f1c411d3dd00d20 | |
| gemma4-12b-b10618 | llama.cppextraction | 97.9% | Coverage 240/240 | g 04e875431e26c4b5d db3addf23a2bc4cec e947bd43665a9bdb | |
| gemma4-12b-ab | llama.cppextraction | 97.9% | Coverage 240/240 | g 04e875431e26c4b5d db3addf23a2bc4cec 8bf8e2bee9ac7333 | |
| gemma4-e2b-ab | llama.cppextraction | 90.4% | Coverage 240/240 | g 04e875431e26c4b5d db3addf23a2bc4cec 23e712860bee6d0e |
Findings3 that name this suite
- 2026-09-17EXL3/QTIP at 4 bpw costs nothing measurable in capability and three quarters of the throughput
- 2026-09-15IQ2_XXS took 27 points off world knowledge and 17 off tool invocation on identical weights
- 2026-09-15MLX and GGUF score the same on a dense model: mean delta 0.55 points over twelve suites