Granite Guardian 4.1 8B · file ← Granite Guardian 4.1 8B
Q4_0
The one we kept, and our judge now. Made here from IBM's file, for speed. Requantised here from IBM's Q4_K_M.
- Size
- 4.8 GB
- Status
- On disk
- Scored runs
- 8
- Smallest Mac
- 24 GB measured, 64K context
| Suite | Score | 95% interval | Passed | Runtime · profile | Recorded |
|---|---|---|---|---|---|
| screening granite-guardian-4.1-8b-b10893-control-nocoh | 88.3% | 77.8–94.2 | Passed 53/60 | llama.cpp extraction | 2026-10-11 |
| screening granite-guardian-4.1-8b-b11552-nocoh | 90.0% | 79.9–95.3 | Passed 54/60 | llama.cpp extraction | 2026-10-11 |
| screening granite-guardian-4.1-8b-q4_0-np1-c64k-b11846 | 95.0% | 86.3–98.3 | Passed 57/60 | llama.cpp extraction | 2026-10-26 |
| screening granite-guardian-4.1-8b-q4_0-np1-c64k-kvq4-b11846 | 90.0% | 79.9–95.3 | Passed 54/60 | llama.cpp extraction | 2026-10-26 |
| screening granite-guardian-4.1-8b-q4_0-np1-c8k-kvq4-b11846 | 90.0% | 79.9–95.3 | Passed 54/60 | llama.cpp extraction | 2026-10-26 |
| screening_native granite-guardian-4.1-8b-q4_0-np1-c64k-b11846 | 90.0% | 79.9–95.3 | Passed 54/60 | llama.cpp extraction | 2026-10-26 |
| screening_native granite-guardian-4.1-8b-q4_0-np1-c64k-kvq4-b11846 | 88.3% | 77.8–94.2 | Passed 53/60 | llama.cpp extraction | 2026-10-26 |
| screening_native granite-guardian-4.1-8b-q4_0-np1-c8k-kvq4-b11846 | 88.3% | 77.8–94.2 | Passed 53/60 | llama.cpp extraction | 2026-10-26 |
Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.
granite-guardian-4.1-8b-q4_0-np1-c64k-b11846
Of record
The measurement of record for this model.
The run’s own note: The live SCREEN seat's Q4_0 file (requantised here from IBM's Q4_K_M), single slot at 64K, f16 KV. A judge model: only screening/1 and screening-native/1 run; the generation suites say nothing about its job. llama-bench (contended): pp512 382.3, tg128 52.9 t/s.
- Server
- 1 slot, 64K of context per request, KV cache f16 (default), flash attention on, 1 suite at a time
- Runtime
- llama.cpp b11846 (1a24df163), plain upstream
- Speed
- decode 51.4 tokens/s, prefill 339.6 tokens/s, medians. Single stream, full fleet resident (contended); short judge prompts and verdicts.
- Peak memory
- 14.65 GB resident, with 64K of context allocated
- Weights
- 4.8 GB file · sha256
d9bdbcc561ade73e… - Coherence check
- not run (judge model: the gate asks for prose, which a judge does not write)
- Left out by plan
- tagger, tagger_nullable, needle, grounding, enrichment, epistemics, synthesis, worldview, nullschema, toolcall, toolcall2, toolscale, tools, contract, chainwalk, shapegen, sitebuild
- Machine
- Mac Studio (M1 Max, 64 GB)
- Released
- 26 October 2026, bundle
2026-10-26-1faf1f39
granite-guardian-4.1-8b-q4_0-np1-c64k-kvq4-b11846
Reference
Kept for reference. It isn't the measurement of record.
The run’s own note: Control run: the same file and build with a q4_0 KV cache, the only change from the run of record. q4_0 KV turns 5 verdicts into prose with no JSON on screening/1.
- Server
- 1 slot, 64K of context per request, KV cache q4_0, flash attention on, 1 suite at a time
- Runtime
- llama.cpp b11846 (1a24df163), plain upstream
- Speed
- decode 47.7 tokens/s, prefill 337.5 tokens/s, medians. Single stream, full fleet resident (contended); short judge prompts and verdicts.
- Peak memory
- 7.46 GB resident, with 64K of context allocated
- Weights
- 4.8 GB file · sha256
d9bdbcc561ade73e… - Coherence check
- not run (judge model: the gate asks for prose, which a judge does not write)
- Left out by plan
- tagger, tagger_nullable, needle, grounding, enrichment, epistemics, synthesis, worldview, nullschema, toolcall, toolcall2, toolscale, tools, contract, chainwalk, shapegen, sitebuild
- Machine
- Mac Studio (M1 Max, 64 GB)
- Released
- 26 October 2026, bundle
2026-10-26-fca5e15e
granite-guardian-4.1-8b-q4_0-np1-c8k-kvq4-b11846
Reference
Kept for reference. It isn't the measurement of record.
The run’s own note: Control run at the live seat's context and KV (8K, q4_0) on b11846. Identical results to the 64K q4_0 run, so context size is not the cause.
- Server
- 1 slot, 8K of context per request, KV cache q4_0, flash attention on, 1 suite at a time
- Runtime
- llama.cpp b11846 (1a24df163), plain upstream
- Speed
- decode 47.7 tokens/s, prefill 337.5 tokens/s, medians. Single stream, full fleet resident (contended); short judge prompts and verdicts.
- Peak memory
- 4.99 GB resident, with 8K of context allocated
- Weights
- 4.8 GB file · sha256
d9bdbcc561ade73e… - Coherence check
- not run (judge model: the gate asks for prose, which a judge does not write)
- Left out by plan
- tagger, tagger_nullable, needle, grounding, enrichment, epistemics, synthesis, worldview, nullschema, toolcall, toolcall2, toolscale, tools, contract, chainwalk, shapegen, sitebuild
- Machine
- Mac Studio (M1 Max, 64 GB)
- Released
- 26 October 2026, bundle
2026-10-26-b5187dee
Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.
| File | granite-guardian-4.1-8b-Q4_0.gguf (made as _q40test/granite-guardian-4.1-8b-Q4_0.gguf) |
| Format | Q4_0 GGUF, 4,824,740,928 B |
| Where from | made here 2026-09-11 13:16Z by llama-quantize --allow-requantize from the Q4_K_M (two rounding passes) |
| Status | on disk; the live SCREEN seat |
How it was made
- Input:
models/granite-guardian-4.1-8b/granite-guardian-4.1-8b-Q4_K_M.gguf(IBM's official file, Granite Guardian 4.1 8B Q4_K_M (IBM official GGUF)). This file is therefore rounded twice (BF16 → Q4_K_M by IBM → Q4_0 here). - Command:
Q=llama.cpp/build/bin/llama-quantize
$Q --allow-requantize granite-guardian-4.1-8b-Q4_K_M.gguf \
models/_q40test/granite-guardian-4.1-8b-Q4_0.gguf Q4_0
Output: "quant size = 4597.83 MiB (4.60 BPW)", 5.86 s.
- Tool:
llama.cppllama-quantize, binaries built 2026-09-10 13:19 local, reporting b10893c32d1dabe(tree HEAD is the local diffusion commit63cbb9227on top) — reconstructed from the binary mtime andllama-server --version, 2026-10-23; the quantize run itself did not print a version. - Date: 2026-09-11 13:16Z (file mtime "11 Sep 14:16" local).
- Size / hash: 4,824,740,928 B; sha256
d9bdbcc561ade73eaed56fb60b4c66ee4e2ebaa43f3aedaca6c392a93093a78a(both computed 2026-10-23). - Who asked / why: Alan, "what would you quant to?". The assistant picked Guardian as "the cleanest candidate … requantising it to Q4_0 isolates format from architecture completely". It was a speed experiment; the assistant said at the time that quality "must be tested before you act on it".
- Moved 2026-09-11 15:51Z from
_q40test/togranite-guardian-4.1-8b/granite-guardian-4.1-8b-Q4_0.gguf.
How it was served
| label / role | dates (UTC) | binary and version | ctx / -np / KV / -fa / --jinja / --reasoning-format | harness profile | why this label exists |
|---|---|---|---|---|---|
| llama-bench only | 2026-09-11 13:16Z, 14:17Z | llama.cpp/build/bin/llama-bench (b10893) | -p 0 -n 64 -ngl 99 -r 2/3 | — | the Q4_0 lever test |
| SCREEN seat (no run-log label) | 2026-09-20 19:24Z → present | .local/bin/llama-server → llama.cpp b10893 c32d1dabe | --jinja -c 8192 -np 1 -ctk q4_0 -ctv q4_0 -fa on -ngl 999 --alias guardian; no reasoning format | — | Alan's "what happened with guardian model"; under launchd from 2026-09-29 (com.freshcontext.seat-guardian) |
granite-guardian-4.1-8b-b10893-control-nocoh | 2026-10-11 ~10:57Z–11:07Z | b10893 c32d1dabe (/props: b10893-c32d1dabe, n_ctx 8192, 1 slot, ftype Q4_0, template sha fed2756d2d24e127) | seat flags as above | extraction, --no-coherence | live-build control for the b11552 upgrade gate (manifest) |
granite-guardian-4.1-8b-b11552-nocoh | 2026-10-11 ~10:57Z–11:08Z | llama.cpp-b11552/build/bin/llama-server, b11552 23b0202a1 | --jinja -c 8192 -np 1 -ctk q4_0 -ctv q4_0 -fa on -ngl 999 --alias guardian | extraction, --no-coherence | the new build under test; the run without --no-coherence refused to score |
Fleetview roster entry: granite-guardian-4.1-8b, upstream model guardian, label "IBM Granite Guardian 4.1 8B Q4_0", note "SCREENER — 100% (20/20/20) · llama.cpp ONLY (36.7% on oMLX) · 8K". It is not in fleet-models.ini.
Decisions
- Made to prove the Q4_0 speed lever on one model, same weights; Alan: "we have a lever, chase it, measure it, prove it".
- Kept 2026-09-11 15:14Z in the "KEEPING (Q4_0 builds, drafters, working MTP)" list and moved into the model dir.
- Became the only Guardian GGUF 2026-09-12 when its parent was deleted under "Q4_K_M with a Q4_0 sibling".
- Seated 2026-09-20 with the parent's 100% in its roster note. That afternoon the assistant had accepted "it gets seated on llama.cpp or re-benched, never assumed". It was seated on llama.cpp; it was not re-benched.
- 2026-10-11 gate: 88.3/90.0 called "identical" across builds; recommendation to switch to b11552. Switch not executed: the symlink still points at b10893 (2026-10-23).
- Not published. A 2026-10-07 inventory ranked the 2026-09-11 Q4_0s as "double-quantised, low value to publish".
What went wrong
- Seated on a borrowed score. The fleetview note, a lab note's SCREEN row and later fleet tables (e.g. "It scored 20/20 on each of keep, fabricated and misattributed") quote 100% (20/20/20) for this file. Its own runs are 20/16/17 and 20/17/17.
- The coherence precondition refused the judge (2026-10-11). Probes like "reply with the single word" got
<think>\n</think>\n<score> no </score>. Membership 67% < 85%, instruction-following 67% < 83%, and the harness said "Retire the artefact or fix the quantisation; do not file the numbers". The runs were then made with--no-coherence. The assistant later logged it as a harness false refusal on specialist models. Whether the Q4_0's format drift and the probe failures share a cause is not measured. - The live seat was used as a benchmark arm (2026-10-11): the b10893 control ran against the production seat.
- Smoke tests show tag looping:
'<think>\n</think>\n<score> no </score></think>\n<score>'on a 16-token budget. The Q4_K_M did the same on 2026-08-31, so this is the model's interface, not the requant. - Missing-flag check:
--jinjapresent; no--reasoning-format. Guardian's think block reachescontent, and that is how the grader reads it. - Stale build stamps: none. Run-log
runtime_versioncomes from /props and matches the binaries.
What happened to it
Requantised here 2026-09-11 13:16Z → speed-benched 2026-09-11 → kept as sole GGUF 2026-09-12 → seated 2026-09-20 → launchd 2026-09-29 → first quality runs 2026-10-11 (88.3 / 90.0) → re-measured 2026-10-26 on b11846 (95.0 with f16 KV; 90.0 with q4_0 KV, the cause of the malformed verdicts) → on disk and serving at models/granite-guardian-4.1-8b/granite-guardian-4.1-8b-Q4_0.gguf (4,824,740,928 B, checked 2026-10-23). Not on HF. Recreating it needs IBM's Q4_K_M plus the command above. A single-pass Q4_0 from the BF16 in the HF cache would be a different artefact.