gpt-oss-20b · file← gpt-oss-20b
gpt-oss-20b MXFP4
The only file we held, a 12.1 GB GGUF with its experts in MXFP4. Tested one morning, then deleted on 11 October. Ggml-org GGUF; deleted 2026-10-11.
- Size
- 12.1 GB
- Status
- Deleted 11 Oct 2026
- Scored runs
- 4
- Smallest Mac
- 24 GB estimate, 8K context
| Suite | Score | 95% interval | Passed | Runtime · profile | Recorded |
|---|---|---|---|---|---|
| nullschemagpt-oss-20b | 72.2% | 61.0–81.2 | Passed 52/72 | llama.cppextraction | 2026-09-12 |
| taggergpt-oss-20b | 86.2% | 81.3–90.0 | Passed 207/240 | llama.cppextraction | 2026-09-12 |
| tagger_nullablegpt-oss-20b99.2% coverage | 85.7% | 80.7–89.6 | Passed 204/238 | llama.cppextraction | 2026-09-12 |
| tagger_nullablegpt-oss-20b-study-default94.3% coverage | 93.9% | 80.4–98.3 | Passed 31/33 | llama.cppstudy-default | 2026-09-12 |
Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.
Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.
| File | gpt-oss-20b-MXFP4.gguf (12,109,566,624 B) |
| Format | MXFP4 (experts) + Q8_0 |
| Where from | downloaded 2026-09-12 |
| Status | benched; deleted 2026-10-11 |
| Hugging Face | ggml-org/gpt-oss-20b-GGUF |
Where it came from
- Source:
ggml-org/gpt-oss-20b-GGUF, filegpt-oss-20b-MXFP4.gguf, pulled with the session'shfpull.shusing 6 parallel streams intomodels/gpt-oss-20b/. Revision not on record; sha256 not on record. The assistant had also listedlmstudio-community/gpt-oss-20b-GGUF. - Size: 12,109,566,624 B; the post-download check printed "size: 12.11 GB (expected 12.11)" and magic GGUF.
- Dates: started 2026-09-12 07:57Z; file complete 08:05Z (listing time 09:05 local).
- Who asked: Alan, "Might as well it's a small enough one to test."
- Tensor mix (read from the GGUF header): MXFP4 10.15 GB (the 72 expert tensors), Q8_0 1.91 GB (token_embd, output.weight, all attention), F32 0.04 GB. The
ftypeshows as unknown (?) in the inventory.
How it was served
| label | dates (UTC) | binary and version | ctx / -np / KV / -fa / --jinja / --reasoning-format | harness profile | why this label exists |
|---|---|---|---|---|---|
gpt-oss-20b | 2026-09-12 08:08-08:27Z (run log local 09:14:31 tagger_nullable, 09:26:34 tagger, 09:27:13 nullschema) | .local/bin/llama-server b10893 (c32d1dabe); manifest b10893-c32d1dabe; run-log 0.4.0-dev (build 10893, commit c32d1dabe) | -c 32768 (manifest: 4 slots, so 8192 per slot) / default 4 / default KV / on / on / not set; -ngl 99 --alias gpt-oss-20b | extraction | same suite versions, dataset, grader and profile as the gemma4-12b baseline |
gpt-oss-20b-study-default | 2026-09-12 ~08:30-08:47Z (run log local 09:47:35) | same server | same | study-default (thinking on, 10k cap) | the arm testing whether deliberation could budge the compliance failures; killed after 35 of 240 cases |
Seats held: none. The server was stopped (pkill) at 08:47Z; the sweep note records no preset, seat or launchd reference to it (a lab note).
Decisions
- Pulled to test a remembered strength against the recorded baseline; verdict: takes no seat, loses to gemma4-12b on all three suites at twice the size.
- Thinking-on arm abandoned: "Yeah kill it" (Alan).
- Marked KEEP in the 2026-09-16 inventory, then deleted on 2026-10-11 (see Lifecycle).
What went wrong
- Killed partial published as a score.
gpt-oss-20b-study-defaulttagger_nullable 93.9% (31/33 of 35 of 240) sits in the run log with void=false and was re-quoted as "tagger_nullable 85.7-93.9" in a lab note. It should be marked partial/void or shown as 35/240. It is also the "easy-first" prefix: the file's first cases, in dataset order, not a sample. - Mean score vs pass rate. The 2026-09-14 table quotes 95.3/97.6/99.0 as if they were percentages comparable with the 85.0/86.2 pass rates.
- First launch failed on the harness interpreter, not the server (
python33.9 vs the harness's 3.12); the server loaded fine. Cost: ~2 minutes. - Self-matching process checks. The assistant twice counted its own
grepas a runningllama-serverand had to re-verify; no effect on the runs. - Missing flags (standing check):
--jinjawas set (needed for the harmony template);--reasoning-formatwas not, andreasoning_charswas 0 for every extraction case. The study-default arm therefore ran with the server's default reasoning handling; its reasoning output was captured (up to 19,800 characters). - Stamp check: the run-log build
10893 / c32d1dabematches the manifest and the transcript's/propsread, so no mis-stamp. - RESTORE entry errors (repo, date, safetensors claim): listed in the family doc.
What happened to it
Downloaded 2026-09-12 07:57-08:05Z -> three suites + partial thinking-on arm 08:08-08:47Z -> inventory KEEP 2026-09-16 -> deleted 2026-10-11 in the sweep, under Alan's approval as recorded in a lab note ("gpt-oss and diffusion models can go"); entry in fleet-roster/models-scripts/RESTORE-deleted-20261011.sh (wrong repo and date; use hf download ggml-org/gpt-oss-20b-GGUF gpt-oss-20b-MXFP4.gguf). The directory is gone (ls models/gpt-oss-20b fails, 2026-10-23). Results provenance survives in benchmarks/results/ (manifests and jsonl present).