FreshContextA context lab · Fife
Drawing
02 · Benchmarks
Sheet
2 of 3
Rev
A · 2026-10-01

Granite 4.2 30B · file← Granite 4.2 30B

Q4_K_M

IBM's GGUF, run on llama.cpp, which held it to the answer format. Deleted on 12 September. IBM GGUF, downloaded.

Size
17.7 GB
Status
Deleted 12 Sep 2026
Scored runs
3
Smallest Mac
32 GB estimate, 8K context
Results3 scored · 95% intervals
SuiteScore95% interval PassedRuntime · profileRecorded
groundinggranite-4.2-30b-gguf84.0%76.6–89.4Passed 105/125llama.cppextraction2026-09-08
nullschemagranite-4.2-30b-gguf90.3%81.3–95.2Passed 65/72llama.cppextraction2026-09-08
screeninggranite-4.2-30b-gguf98.3%91.1–99.7Passed 59/60llama.cppextraction2026-09-08

Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.

The recordHow it was made · how it was served · what went wrong · what happened to it

Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.

Filegranite-4.2-30b-Q4_K_M.gguf
FormatGGUF Q4_K_M, 17,721,455,328 B
Where fromdownloaded 2026-09-07 from IBM's GGUF repo
Statusdeleted 2026-09-12 07:45Z (Alan's instruction)
Hugging Facehttps://huggingface.co/ibm-granite/granite-4.2-30b-GGUF

Where it came from

  • Source: ibm-granite/granite-4.2-30b-GGUF, file granite-4.2-30b-Q4_K_M.gguf.
  • Revision and sha256: not on record. The download used hf download without a pin; the HF metadata under .cache was removed at 22:07Z by rm -rf …/.cache and the folder on 2026-09-12.
  • Command, run from models:
HF_HUB_ENABLE_HF_TRANSFER=1 HF_XET_HIGH_PERFORMANCE=1 hf download ibm-granite/granite-4.2-30b-GGUF granite-4.2-30b-Q4_K_M.gguf --local-dir granite-4.2-30b-gguf

It waited for tagger_fair to finish so the download would not disturb a live measurement, started at 21:20:17Z, was paused at 5.4 GB at 21:27Z to free bandwidth and resumed by a one-off script at 21:43:31Z, using the bfcl-venv copy of hf.

  • Not a corrupt download: the folder read 22 GB against 16.50 expected; 5.6 GB of it was stale .cache from the interrupted first attempt, removed at 22:07Z.
  • Who asked: Alan pasted IBM's blurb at 21:00Z; the assistant recommended pulling it. Alan answered "yes" at 21:02Z ((reconstructed) as approval). Why Q4_K_M and not another quant: not on record (the card listed Q4_K_M, Q5_K_M, Q6_K and Q8_0).

How it was served

labeldates (UTC)binary and versionctx / -np / KV / -fa / --jinja / --reasoning-formatharness profilewhy this label exists
granite-4.2-30b-gguf2026-09-07 about 23:15Z to 23:29Z (run-log finish times 23:22:10, 23:26:42, 23:28:43Z)llama.cpp/build/bin/llama-server, 0.2.0-dev (build 10618, commit eb25b7263); the tree sat at that commit from 2026-08-25 until the 2026-09-10 pull (reflog)-c 16384 / -np not set / KV not set (f16) / -fa not set / --jinja on / --reasoning-format not set; -ngl 999 --alias granite-4.2-30b-ggufextraction (temperature 0, thinking false), screening, grounding, nullschemathe GGUF half of the controlled runtime pair: "same weights, same bit depth, same suites, only the runtime differs"
granite-4.2-30b (contract row only; the same label's toolcall row is in Granite 4.2 30B MLX 4-bit (IBM, downloaded))2026-09-08 19:28Z, finished 19:31:49Z (run-log 20:31:49 BST)same binary and commit-c 8192 / -np not set / KV not set / -fa not set / --jinja on / --reasoning-format not set; -ngl 999 --alias granite-4.2-30bextraction, contract/1cheap control for Alan's system-prompt question, run beside gemma4-12b and REAP: "we need a system prompt to see if models adhere to it"

Seats held: none. The first overnight script (queued 22:56Z) targeted the same port and trigger as the compute-chain script; it was killed at 23:03Z before it ran, so only the compute-chain run exists.

Other servings that are not run-log labels (all from llama.cpp, same commit):

purposewhen (UTC)flags
sentinel schema-enforcement probe, alias gr2026-09-08 10:26Z--jinja -c 8192 -ngl 999
context ladder (8k to 131k), eval/ctx_ladder.py2026-09-08 about 10:58-11:03Z--n-gpu-layers 99 --ctx-size N --jinja, one server per rung
runtime matrix, llama.cpp leg, alias granite2026-09-08 about 13:45-14:39Z--jinja -c 16384 -ngl 999
sequential-read timing of the first 4.29 GB of this file (an SSD benchmark that borrowed it, not a model run)2026-09-08 11:30Znone

Decisions

  • Why downloaded: Alan's interest in a dense Apache-2.0 30B with structured output, plus the assistant's case that nullschema was the fleet's weakest axis. It was the GGUF baseline for the MLX pair.
  • Why kept: it was the only valid half of the pair (MLX could not enforce schemas), so the GGUF carried the granite result. It was listed as "benched" in the 2026-09-08 inventory and stayed on disk until 2026-09-12.
  • Why never seated: no sourced reason from Alan. The record shows 9 tok/s decode, a nullschema score not separable from gemma4-12b at 7 GB, and qwen3-next-80b faster at 37.7 tok/s. (reconstructed)
  • Why deleted: Alan, 2026-09-12 07:45Z: "I don't think we’re gonna use granite 30b kill it". Executed in one rm -rf of the folder.
  • Published: yes, as write-up granite-4.2-30b-gguf and in The extraction suites can no longer separate the leading models and Schema enforcement is per-model, not per-runtime.

What went wrong

  • Failed first contract run. At 19:27Z contract_run.sh died with TypeError... unexpected keyword argument 'system' because a runner patch had not applied; it wrote no rows and was killed. A zero-byte contract__granite-4.2-30b__extraction.jsonl.superseded remains. The 19:28Z rerun is the one recorded.
  • Memory false alarm during that run ("6.2 GB reclaimable" read as exhaustion) turned out to be evictable page cache, and led to eval/tools/memfree.py.
  • Stale build stamps: none. All llama.cpp runs report eb25b7263, matching the reflog.
  • Missing flags: --jinja was on for every llama.cpp serving. --reasoning-format was never set; the probe's response carried a separate reasoning_content field, so the default split reasoning from content. -fa and -np were left at build defaults.
  • Partial runs: none; row counts equal case counts.

What happened to it

Downloaded 2026-09-07 21:20Z to about 22:07Z → benched 2026-09-07/08 → ladder, matrix and contract 2026-09-08 → listed as a duplicate format 2026-09-11 → deleted 2026-09-12 07:45Z. No RESTORE-deleted-*.sh entry. Re-downloadable: ibm-granite/granite-4.2-30b-GGUF/granite-4.2-30b-Q4_K_M.gguf, 17,721,455,328 bytes. Status: retired.