MLX 4-bit with MTP sidecar

Where it came from

• Repo: rapid-mlx/Qwen3.8-27B-4bit-MTP-MLX, revision aa985c29ff5b334cbfdcbbc787d47e66e9d9e456 (HF cache refs/main, read 2026-10-23). Alias in rapid-mlx: qwen3.8-27b-4bit.
• Date and tool: 2026-09-09 07:03Z, rapid-mlx pull rapid-mlx/Qwen3.8-27B-4bit-MTP-MLX with HF_XET_HIGH_PERFORMANCE=1; 15.2 GiB in 9m 13s; cache files dated 07:03–07:12Z.
• Contents (stat of the cache, 2026-10-23): three shards of 5,343,268,662 + 5,354,185,130 + 5,357,087,557 = 16,054,541,349 B of weights (blob names are the sha256: 6cc1508e…, 83f2a20c…, 31b8c91e…); a separate mtp/model.safetensors sidecar of 238,934,137 B (76663c10…, with a .sha256 file beside it); tokenizer.json 19,989,325 B; vocab.json 6,722,759 B; a real chat_template.jinja of 8,952 B with reasoning_effort defaulting to xhigh. rapid-mlx's README expects a colocated model-mtp.safetensors, but the repo ships the sidecar at mtp/model.safetensors, and its MTP wiring only fires for registered aliases, not for the raw repo id.
• Wrapper made here: models/qwen3.8-27b-mtp-medium, 2026-09-09 07:43Z, a directory of symlinks to the three cache shards plus copied small files, with one sed changing the template's default from xhigh to medium. The medium default is a template-level change, not a retrain; the weights are the curated build's.
• Who asked: Alan, 06:53Z–07:42Z.

How it was served

The rapid-mlx re-validation of toolcall/1 on this build (12 of 12 on every arm) has no run-log row of its own: the shared results file toolcall__qwen3.8-27b__extraction.jsonl was overwritten by the oMLX run (mtime 2026-09-09 11:56 BST).

Seats held: the 'Agent / assistant seat' and on 2026-09-09 (assistant's role table at 10:45Z, the seat mlx_lm qwen3.8-27b agent seat at 14:35Z). By 2026-09-10 12:29Z the fleet table has Qwen3.8 on the GGUF. When the 9 September MLX seat stopped is not on record (would have been in the session around 2026-09-10 06:50Z; it is described as 'already downloaded' at that time).

Decisions

• Why downloaded: to test whether the curated build was the 41 tok/s recipe and whether its MTP sidecar helped. It was not: the speed was a wash and the sidecar never engaged under rapid-mlx.
• Why wrapped with medium: Alan's requirement above; the assistant found the template default is xhigh and that rapid-mlx ignores the template, so the patched directory was served through mlx_lm and oMLX, which do honour it.
• Why it was the agent seat: the only model to reach for a tool nobody mentioned (12 of 12 against 1 of 12 and 0 of 12). The tool-use finding, Tool perception is binary and tracks the checkpoint, not size or vendor claims, is stated against the checkpoint, not the quant.
• Why it stopped being the seat (reconstructed): K2-Horizon-36B tied it 12/12 on every toolcall arm at 3.6 times the speed on 2026-09-10, and Alan had said of K2 "do it, this could become my daily driver" on 2026-09-09. The A/B in Qwen3.8-27B UD-Q4_K_M (unsloth GGUF, MTP head embedded) then moved the Qwen3.8 role to 'writes'. On 2026-10-09 the roles read: write is qwen3.8-27b, act is k2-36b.
• Published: as the qwen3.8-27b model write-up (quant text 'MLX 4-bit (MTP build, template patched to reasoning_effort=medium)').

What went wrong

• 'Broken download' is a symlink mis-read. On 2026-09-11 the assistant reported qwen3.8-27b-mtp-medium/ as 'a broken download' whose three shards were '157-byte LFS pointers, never fetched' (also in the fleet_audit.py docstring). The 157 bytes are the length of the symlink target path (.cache/huggingface/hub/models--rapid-mlx--Qwen3.8-27B-4bit-MTP-MLX/blobs/<64 hex> is exactly 157 characters), ls -l reports a symlink's size that way, and du without -L showed 26 MB; the cache blobs behind them are complete (above). The directory had been served successfully on 2026-09-09. The build was also not 'never run'. Cost: an inventory entry marked broken and a model counted as unavailable.
• rapid-mlx drops the model's chat template and chat_template_kwargs (a bogus kwarg returns 200 instead of an error; identical prompt tokens across effort levels).
• Missing flags: none recorded for the oMLX run; its server flags are not on record (oMLX discovers models at start).

What happened to it

Pulled 2026-09-09 07:03Z → wrapped 07:43Z → served and benched 2026-09-09 → seat displaced 2026-09-09/10 → HF cache snapshot still on disk (.cache/huggingface/hub/models--rapid-mlx--Qwen3.8-27B-4bit-MTP-MLX, read 2026-10-23); the wrapper directory models/qwen3.8-27b-mtp-medium no longer exists, and when it was removed is not on record. labs-control.sh still references the old path. Re-obtainable with rapid-mlx pull rapid-mlx/Qwen3.8-27B-4bit-MTP-MLX plus the one-line template edit.