Google QAT Q4_0 GGUF

Where it came from

• Repo / file: google/gemma-4-E2B-it-qat-q4_0-gguf / gemma-4-E2B_q4_0-it.gguf, listed at 3.35 GB on 2026-08-23.
• Command: models/gemma4-e2b/get.sh, written 2026-08-23 10:19Z and run in the background batch at 10:19Z:

cd models/gemma4-e2b
B=https://huggingface.co/google/gemma-4-E2B-it-qat-q4_0-gguf/resolve/main
for f in gemma-4-E2B_q4_0-it.gguf gemma-4-E2B-it-mmproj.gguf; do
$PG "$B/$f" "$f" 6 # pget.sh, 6 parallel range requests
done
print DONE > .dl_complete

• Revision: not pinned (resolve/main); not on record.
• Size: 3,349,516,256 B (stat, 2026-10-23). sha256 not on record.
• Date: completed 2026-08-23 ~10:47Z (file mtime "23 Aug 11:47" local; .dl_complete 11:48). Verified on disk with the rest of the batch at 11:01Z ("gemma4-e2b 2 files 4.3 GB").
• Who asked: Alan, "also want to try gemma4 smaller models at full context".
• Format note: this is Google's quantisation-aware-trained Q4_0, not a post-training quant. Asked on 2026-09-11 what he should requantise E2B to, the answer was that there is "no better format to move it to". The GGUF header reports 4.63 B parameters and 3.10 GiB (llama-bench).
• Sibling file from the same script: Gemma 4 E2B — vision projector GGUF (gemma-4-E2B-it-mmproj.gguf).

How it was served

All llama.cpp servers below were llama.cpp/build/bin/llama-server (b10369 6e62ba538 until the 2026-08-25 07:30Z rebuild to b10618 eb25b7263), except the 2026-10-05 run (.local/bin/llama-server → same tree, /props b10893-c32d1dabe). None set --reasoning-format, a KV type or (before October) -fa; all set --jinja -ngl 99 -cb.

Per-run serving for gemma4-e2b (run finish UTC):

Profiles (from the manifests): study-nothink = thinking off, max_tokens 10000 as a safety ceiling; agentic-fast = thinking on, reasoning budget 512, max_tokens 4096; extraction = thinking off, max_tokens 2048.

Not served by this file: the gemma4-e2b toolcall/1 run of 2026-09-09 (oMLX, MLX community file — see Gemma 4 E2B — mlx-community MLX 4-bit (gemma-4-e2b-it-4bit)).

Seats held: none. No fleet-models.ini preset, launchd plist or fleetview entry for this file exists in fleet-roster git history or on disk (checked 2026-10-23). It was recommended as a fourth seat "at 8k" on 2026-09-13 and as the tagger in a four-seat plan on 2026-09-14; neither was launched.

Decisions

• Downloaded for Alan's small-Gemma full-context question.
• Kept through every sweep: not in the 2026-10-11 tiers (a lab note); still on disk.
• Not requantised (2026-09-11): already QAT Q4_0.
• Not chosen as extractor (2026-08-25, F1 gap to the 12B), as tagger (the 12B took the TAG seat) or as provenance seat (2026-10-05).
• Not published to HF (it is Google's file).

What went wrong

• Blended epistemics file (2026-08-23 23:11Z). "gemma4-e2b shows 96 cases in a 48-case suite" — --no-resume appended rather than truncated; fixed so --no-resume supersedes the old file.
• epistemics/3 → /4 swing. Output-format wording alone moved E2B 85.4 → 68.8. The board shows only /4.
• First grid run died on a drift refusal for E2B epistemics and was re-run; /health returned 200 while models were still loading, so readiness switched to a one-token completion.
• 2026-10-05 first attempt produced nothing: a relative venv path and a coherence.py TypeError; re-run cleanly a minute later.
• Inflated tagger figures in conversation (98.5 / 98.4 from the partial-credit score mean, 2026-09-14; 97.9 misattributed 2026-09-12; GGUF/MLX mixed 2026-09-20) — see the family doc. The run log was never wrong; the quoted numbers were.

What happened to it

Downloaded 2026-08-23 → benched 2026-08-23 → 2026-10-05 (four llama.cpp builds' worth of labels: b10369, b10618, b10893) → never seated → on disk at models/gemma4-e2b/gemma-4-E2B_q4_0-it.gguf (3,349,516,256 B, checked 2026-10-23). Re-downloadable from google/gemma-4-E2B-it-qat-q4_0-gguf with get.sh. No RESTORE entry needed (never deleted).