Granite Guardian 4.1 8B · file← Granite Guardian 4.1 8B

Q4_0

The one we kept, and our judge now. Made here from IBM's file, for speed. Requantised here from IBM's Q4_K_M.

Size: 4.8 GB
Status: On disk
Scored runs: 2
Smallest Mac: 16 GB estimate, 8K context

Results2 scored · 95% intervals

Suite	Score	95% interval	Passed	Runtime · profile	Recorded
screeninggranite-guardian-4.1-8b-b10893-control-nocoh	88.3%	77.8–94.2	Passed 53/60	llama.cppextraction	2026-10-11
screeninggranite-guardian-4.1-8b-b11552-nocoh	90.0%	79.9–95.3	Passed 54/60	llama.cppextraction	2026-10-11

Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.

The recordHow it was made · how it was served · what went wrong · what happened to it

Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.

File	`granite-guardian-4.1-8b-Q4_0.gguf` (made as `_q40test/granite-guardian-4.1-8b-Q4_0.gguf`)
Format	Q4_0 GGUF, 4,824,740,928 B
Where from	made here 2026-09-11 13:16Z by `llama-quantize --allow-requantize` from the Q4_K_M (two rounding passes)
Status	on disk; the live SCREEN seat

How it was made

Input: models/granite-guardian-4.1-8b/granite-guardian-4.1-8b-Q4_K_M.gguf (IBM's official file, Granite Guardian 4.1 8B Q4_K_M (IBM official GGUF)). This file is therefore rounded twice (BF16 → Q4_K_M by IBM → Q4_0 here).
Command:

Q=llama.cpp/build/bin/llama-quantize
$Q --allow-requantize granite-guardian-4.1-8b-Q4_K_M.gguf \
models/_q40test/granite-guardian-4.1-8b-Q4_0.gguf Q4_0

Output: "quant size = 4597.83 MiB (4.60 BPW)", 5.86 s.

Tool: llama.cpp llama-quantize, binaries built 2026-09-10 13:19 local, reporting b10893 c32d1dabe (tree HEAD is the local diffusion commit 63cbb9227 on top) — reconstructed from the binary mtime and llama-server --version, 2026-10-23; the quantize run itself did not print a version.
Date: 2026-09-11 13:16Z (file mtime "11 Sep 14:16" local).
Size / hash: 4,824,740,928 B; sha256 d9bdbcc561ade73eaed56fb60b4c66ee4e2ebaa43f3aedaca6c392a93093a78a (both computed 2026-10-23).
Who asked / why: Alan, "what would you quant to?". The assistant picked Guardian as "the cleanest candidate … requantising it to Q4_0 isolates format from architecture completely". It was a speed experiment; the assistant said at the time that quality "must be tested before you act on it".
Moved 2026-09-11 15:51Z from _q40test/ to granite-guardian-4.1-8b/granite-guardian-4.1-8b-Q4_0.gguf.

How it was served

label / role	dates (UTC)	binary and version	ctx / -np / KV / -fa / --jinja / --reasoning-format	harness profile	why this label exists
llama-bench only	2026-09-11 13:16Z, 14:17Z	`llama.cpp/build/bin/llama-bench` (b10893)	`-p 0 -n 64 -ngl 99 -r 2/3`	—	the Q4_0 lever test
SCREEN seat (no run-log label)	2026-09-20 19:24Z → present	`.local/bin/llama-server` → `llama.cpp` b10893 `c32d1dabe`	`--jinja -c 8192 -np 1 -ctk q4_0 -ctv q4_0 -fa on -ngl 999 --alias guardian`; no reasoning format	—	Alan's "what happened with guardian model"; under launchd from 2026-09-29 (`com.freshcontext.seat-guardian`)
`granite-guardian-4.1-8b-b10893-control-nocoh`	2026-10-11 ~10:57Z–11:07Z	b10893 `c32d1dabe` (/props: `b10893-c32d1dabe`, n_ctx 8192, 1 slot, ftype Q4_0, template sha `fed2756d2d24e127`)	seat flags as above	extraction, `--no-coherence`	live-build control for the b11552 upgrade gate (manifest)
`granite-guardian-4.1-8b-b11552-nocoh`	2026-10-11 ~10:57Z–11:08Z	`llama.cpp-b11552/build/bin/llama-server`, b11552 `23b0202a1`	`--jinja -c 8192 -np 1 -ctk q4_0 -ctv q4_0 -fa on -ngl 999 --alias guardian`	extraction, `--no-coherence`	the new build under test; the run without `--no-coherence` refused to score

Fleetview roster entry: granite-guardian-4.1-8b, upstream model guardian, label "IBM Granite Guardian 4.1 8B Q4_0", note "SCREENER — 100% (20/20/20) · llama.cpp ONLY (36.7% on oMLX) · 8K". It is not in fleet-models.ini.

Decisions

Made to prove the Q4_0 speed lever on one model, same weights; Alan: "we have a lever, chase it, measure it, prove it".
Kept 2026-09-11 15:14Z in the "KEEPING (Q4_0 builds, drafters, working MTP)" list and moved into the model dir.
Became the only Guardian GGUF 2026-09-12 when its parent was deleted under "Q4_K_M with a Q4_0 sibling".
Seated 2026-09-20 with the parent's 100% in its roster note. That afternoon the assistant had accepted "it gets seated on llama.cpp or re-benched, never assumed". It was seated on llama.cpp; it was not re-benched.
2026-10-11 gate: 88.3/90.0 called "identical" across builds; recommendation to switch to b11552. Switch not executed: the symlink still points at b10893 (2026-10-23).
Not published. A 2026-10-07 inventory ranked the 2026-09-11 Q4_0s as "double-quantised, low value to publish".

What went wrong

Seated on a borrowed score. The fleetview note, a lab note's SCREEN row and later fleet tables (e.g. "It scored 20/20 on each of keep, fabricated and misattributed") quote 100% (20/20/20) for this file. Its own runs are 20/16/17 and 20/17/17.
The coherence precondition refused the judge (2026-10-11). Probes like "reply with the single word" got <think>\n</think>\n<score> no </score>. Membership 67% < 85%, instruction-following 67% < 83%, and the harness said "Retire the artefact or fix the quantisation; do not file the numbers". The runs were then made with --no-coherence. The assistant later logged it as a harness false refusal on specialist models. Whether the Q4_0's format drift and the probe failures share a cause is not measured.
The live seat was used as a benchmark arm (2026-10-11): the b10893 control ran against the production seat.
Smoke tests show tag looping: '<think>\n</think>\n<score> no </score></think>\n<score>' on a 16-token budget. The Q4_K_M did the same on 2026-08-31, so this is the model's interface, not the requant.
Missing-flag check: --jinja present; no --reasoning-format. Guardian's think block reaches content, and that is how the grader reads it.
Stale build stamps: none. Run-log runtime_version comes from /props and matches the binaries.

What happened to it

Requantised here 2026-09-11 13:16Z → speed-benched 2026-09-11 → kept as sole GGUF 2026-09-12 → seated 2026-09-20 → launchd 2026-09-29 → first quality runs 2026-10-11 (88.3 / 90.0) → on disk and serving at models/granite-guardian-4.1-8b/granite-guardian-4.1-8b-Q4_0.gguf (4,824,740,928 B, checked 2026-10-23). Not on HF. Recreating it needs IBM's Q4_K_M plus the command above. A single-pass Q4_0 from the BF16 in the HF cache would be a different artefact.