mlx-community MLX 4-bit

Where it came from

• Repo: mlx-community/gemma-4-e2b-it-4bit (community conversion, not Google's). Quantisation per its config.json: {'group_size': 64, 'bits': 4, 'mode': 'affine'}, model_type gemma4, architecture Gemma4ForConditionalGeneration.
• Commands (2026-08-23 20:59Z and 21:01Z):

cd models && mkdir -p gemma4-e2b-mlx-4bit && cd gemma4-e2b-mlx-4bit
B=https://huggingface.co/mlx-community/gemma-4-e2b-it-4bit/resolve/main
for f in config.json tokenizer.json tokenizer_config.json generation_config.json chat_template.jinja model.safetensors.index.json; do
curl -sL -o "$f" "$B/$f"
done
$PG "https://huggingface.co/mlx-community/gemma-4-e2b-it-4bit/resolve/main/model.safetensors" model.safetensors 6

• Why: to be the exact base of Alan's LoRA tagger, so the tuned-vs-base A/B had "tuning as the only variable". A lab note records mlx-community/gemma-4-e2b-it-4bit as the student base.
• Not a local conversion. The void note and run-log void_reason call it "a local MLX 4-bit convert"; the record shows a download of the community conversion.

How it was served

Unlabelled uses (not in the run log): in-process mlx_lm.generate on 2026-08-23 (base-e2b, 59 turns); mlx_lm.perplexity on 2026-09-08; its tokenizer was pointed at by a BFCL handler registration for both E2B and E4B on 2026-08-24 (no BFCL run is on record).

Seats held: none. On 2026-09-09 10:35Z it was put "straight into the oMLX pool for tagging"; oMLX did not discover it at first ("Model 'gemma4-e2b' not found") and it was never a fleet seat.

Decisions

• Downloaded 2026-08-23 as the controlled base for the tuned tagger.
• Used for the fine-tune verdict 2026-09-07: base 85.8 [81–90] vs distill 74.2 [68–79] on tagger, "Distill verdict: don't cut over" (The tagger fine-tune scored below its own base model). The assistant first read the 85.8 as the GGUF's 91.2 being "config-dependent"; the two are different files on different runtimes.
• Voided 2026-09-09: the oMLX toolcall 0/60 "is not a finding about gemma" — oMLX plus an upstream-reported broken Gemma 4 MLX 4-bit (mlx-vlm #895, cited); voided with the other oMLX runs at 17:33Z. The responses are real direct answers (answered_directly on every case), not server errors (results file .void-omlx, read 2026-10-23).
• Marked unusable without SwiftLM/Xcode in the 2026-09-07 disk triage ("The MLX cluster"... "that's unusable without Xcode and SwiftLM") — a claim the same evening's mlx_lm server runs contradicted.
• Deleted 2026-10-11 in the Tier A sweep ("nuke tiers a & b", quoted in RESTORE-deleted-20261011-tiersAB.sh), as "superseded by the live QAT GGUF TAG seat" (a lab note). No E2B GGUF seat ever existed; the TAG seat is gemma4-12b.

What went wrong

• Void oMLX run under the GGUF's label. The only toolcall/1 run for E2B served this file under gemma4-e2b, so the label mixes two weights files. The run-log row's llama_build says 0.2.0-dev (build 10618, commit eb25b7263) though oMLX served it — a stale build stamp.
• "Local convert" in the void reason — it was a download.
• RESTORE entry has no re-download command. RESTORE-deleted-20261011-tiersAB.sh lists it in a comment under the ternary-bonsai block, whose only re-download line is for PrismML; the source is mlx-community/gemma-4-e2b-it-4bit.

What happened to it

Downloaded 2026-08-23 21:04Z → probed 2026-08-23 (59-turn A/B), 2026-09-07 (run log), 2026-09-08 (perplexity), 2026-09-09 (oMLX, void) → deleted 2026-10-11 (the restore script lists it but has no download command) → re-downloadable with hf download mlx-community/gemma-4-e2b-it-4bit (bytes may differ if the repo has changed; revision not recorded). Retired variant (no longer held).