QAT Q4_0

Where it came from

• Repo / file: google/gemma-4-12B-it-qat-q4_0-gguf / gemma-4-12b-it-qat-q4_0.gguf. The HF listing on 2026-08-23 gave 6.98 GB.
• Command:

B=https://huggingface.co/google/gemma-4-12B-it-qat-q4_0-gguf/resolve/main
for f in gemma-4-12b-it-qat-q4_0.gguf mmproj-gemma-4-12b-it-qat-q4_0.gguf; do
pget.sh "$B/$f" "$f" 6
done

It ran in the background with the Qwen3.5-9B and Gemma E4B/E2B pulls. Finished: file mtime 23 Aug 11:37:39 BST (10:37Z), with .dl_complete written 11:37 BST.

• Revision: not on record (resolve/main, unpinned). sha256: not on record.
• Size: 6,975,879,296 B (stat, 2026-10-23). Older docs' "6.5 GB" is GiB.
• Quant: Q4_0 with Google's quantisation-aware training. /props reports model_ftype: Q4_0 for this path (manifests of gemma4-12b-recheck and -b11552). The local GGUF header was not read.
• Who asked / why: Alan, 2026-08-23 10:16Z: "also want to try gemma4 smaller models at full context".

How it was served

Seats held (none came from fleet-models.ini: the 12B was never a router preset, and git log -p on the ini has no gemma4-12b line):

• 2026-09-09 16:45Z → 2026-09-12: the seat --mmproj -c 8192 -np 2 (tagger, then Hermes compression/approval/vision aux roles, 2026-09-10).
• 2026-09-12 09:55Z: the seat vision-only (vision-seat-gemma.sh, -c 16384), killed 09:57Z.
• 2026-09-12 10:10Z: the seat reasoning + tagger (reasoning-seat-gemma12b.sh, -c 65536 -np 1), served zero requests, down by 09-13 19:16Z.
• 2026-09-13 20:44Z: the seat -c 131072 -np 2 --cache-ram 4096 -ctk q4_0 -ctv q4_0 -fa on --mmproj.
• 2026-09-20 17:50Z → present: the seat TAG worker (fleetview ROSTER, thinking pinned off). -c 8192 -np 1 -ctk q4_0 -ctv q4_0 -fa on -ngl 999 --jinja, no mmproj. 2026-09-20 22:04Z → 09-29 19:54Z: -c 32768 -np 4. After the segfault: hand relaunch at 1×8K, then a launchd job with the same 1×8K args. Binary: .local/bin/llama-server → llama.cpp (reports b10893 c32d1dabe; HEAD is 63cbb9227 on fc/diffusion-confidence).

Decisions

• Downloaded as one of the "gemma4 smaller models".
• Kept at QAT Q4_0 when higher quants were offered. Alan: "nah qat stays"; "it's what makes gemma as good as it is now, it's been retuned".
• Kept in the 2026-09-08 sweep, where Alan's "Gemma dense" was read as the 31B.
• Seated as the TAG worker 2026-09-20, and supervised by launchd from 2026-09-29.
• Not published to HF: it is Google's file, unmodified.

What went wrong

• Partial tagger run with a wrong published count (gemma4-12b tagger/2, study-nothink). At 23:19Z on 2026-08-23 the assistant launched the tool grid on the same port. Its first model (gemma4-e2b, logged "00:19:34" BST) replaced the 12B's server mid-run. The tagger file's final write is exactly 00:19:34 BST. Rows 193–240 are URLError: Connection refused (45), plus one RemoteDisconnected and one ConnectionResetError. The file was then rewritten by hand to drop 503 rows and dedupe by case id. It now has 240 rows on 239 physical lines: line 127 holds both tag/tp034 and tag/tp035. The run log reads 238 attempted / 191 measured / 178 passed (93.2%, coverage 80.3%). The file holds 240 / 193 / 180 (93.3%, coverage 80.4%). Cost: a 47-case hole in the first tagger number. It was superseded in practice by the full 240/240 -ab run the next day.
• Blended files from harness bugs on 2026-08-23, all found the same night (a lab note §5):
• the /health readiness bug left 48 HTTP-503 rows in the epistemics file;
• --no-resume appended instead of truncating, leaving 339 tagger rows for a 240-case suite;
• the | template penalised the 12B for following format literally (epistemics/3 answerable 9/12, fixed in /4).
• Empty smoke-test replies, twice. On 2026-08-31 the gate's max_tokens=16 probe returned '' for gemma, a false alarm. On 2026-09-20 an 8-token seating probe returned '' because thinking consumed the budget, so thinking is pinned off in fleetview.
• Metal segfault, 2026-09-29 19:54Z (20:54:44 BST): null pipeline in ggml_metal_op_flash_attn_ext after 9 days and 2.74M tasks. Down until a hand relaunch at 20:31Z; the root cause is not established. It led to launchd supervision.
• Borrowed evidence in the roster note. fleetview's note "np-insensitive: 76.7 vs 76.9 measured" is MiniCPM5-2B's figure (One baseline was 25 points off and nothing in the stamp could explain it; the other six reproduced exactly). It was cited for gemma when the seat went to 4 slots. No gemma -np A/B on tagger is on record.
• Missing flags check: every server line above has --jinja. The -midrange server added --reasoning-format deepseek. The others leave reasoning parsing at the binary's default; the effect on thinking-off profiles is not on record.

What happened to it

Downloaded 2026-08-23 → benched 2026-08-23 → 2026-10-11 (32 run-log entries, 10 labels) → seated on and off 2026-09-09 → 09-13 → permanent TAG seat from 2026-09-20, launchd from 2026-09-29 → on disk at models/gemma4-12b-dense/gemma-4-12b-it-qat-q4_0.gguf (6,975,879,296 B, checked 2026-10-23). Never deleted, so no RESTORE entry. Re-downloadable from google/gemma-4-12B-it-qat-q4_0-gguf (revision unpinned).