Qwen3-Next-80B-A3B-Instruct REAP · file← Qwen3-Next-80B-A3B-Instruct REAP

MLX 4-bit, converted 2026-09-08

Our first MLX copy, at 4 bits. Two tests, then deleted for the mixed copy.

Size: 32.0 GB
Status: Deleted 9 Sep 2026
Scored runs: 2
Smallest Mac: 64 GB estimate, 8K context

Results2 scored · 95% intervals

Suite	Score	95% interval	Passed	Runtime · profile	Recorded
contractqwen3-next-80b-reap-mlx-q4	47.2%	32.0–63.0	Passed 17/36	mlx_lmextraction	2026-09-08
screeningqwen3-next-80b-reap-mlx-q4	100.0%	94.0–100.0	Passed 60/60	mlx_lmextraction	2026-09-08

Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.

Corrections4 to the record published 21 October 2026

4 runs voided (the reapq4-mlx_lm contract run of 8 September 2026; the reapq4-mlx_lm screening run of 8 September 2026; the reapq4-rapid-mlx contract run of 8 September 2026; the reapq4-rapid-mlx screening run of 8 September 2026). Every request failed (HTTP 404: the label was sent as the model id), so nothing was measured.
Corrected 24 October 2026 · a mechanical fix

The recordHow it was made · how it was served · what went wrong · what happened to it

Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.

File	`qwen3-next-80b-reap-mlx-q4`, ~32 GB
Format	MLX 4-bit g64, made here
Where from	`mlx_lm convert` of the safetensors, 2026-09-08
Status	deleted 2026-09-09 14:08Z
Hugging Face	not on HF

How it was made

Input: the Samsung safetensors (Qwen3-Next-80B-A3B-Instruct REAP safetensors source (SamsungSAILMontreal)), first copy (models/reap-src).
Command: eval/reap_mlx.sh: mlx_lm convert --hf-path reap-src --mlx-path qwen3-next-80b-reap-mlx-q4 -q --q-bits 4 --q-group-size 64, run from the stale mlx-venv (the assistant later noted the conversions came from an older mlx than the one that benchmarked them). The run was a stage of a one-off script, followed by the mixed build from the same source.
Tool and version: mlx_lm version at conversion not on record (0.31.3 is what benchmarked it).
Date and size: completed by 2026-09-08 20:06Z; du -sm 32,465 MB, "32 GB". Shard count and exact bytes: not on record (would have been in ls -l).
Recipe detail not recorded at the time: the 2026-09-10 reconversion showed that mlx_lm.convert applies 96 router overrides at 8-bit even at default settings, so this build almost certainly carried the same overrides; reconstructed, not read from this build's config (the directory was deleted unread).
Who asked: Alan, 19:02Z: "So reap mlx is next then". The assistant had proposed it twice.

How it was served

label	dates (UTC)	binary and version	ctx / -np / KV / -fa / `--jinja` / `--reasoning-format`	harness profile	why this label exists
`reapq4-rapid-mlx`	2026-09-08 20:07:11Z (finish 21:07 BST)	`rapid-mlx` 0.13.4, `serve <path> --no-spec-decode`	n/a (MLX)	extraction	first attempt; the label was used as the wire id, so every case was excluded HTTP 404, 0 measured
`reapq4-mlx_lm`	2026-09-08 20:09:32Z to 20:09:36Z	`mlx_lm` 0.31.3, `server --model <path>`	n/a	extraction	same 404 fault
`qwen3-next-80b-reap-mlx-q4`	2026-09-08 20:41:43Z screening, 20:42:32Z contract (finish 21:41 to 21:42 BST)	rapid-mlx first, then `mlx_lm` 0.31.3 (the run log keeps the mlx_lm row)	`--model-id <path> --no-resume`, wire id = model path	extraction	corrected re-run: the bench order was q4 on rapid-mlx, mixed36 on rapid-mlx, q4 on mlx_lm

Seats held: none.

Decisions

Why made: to give REAP "under all three runtimes" and to see whether MLX could lift the 41.4 tok/s llama.cpp figure toward the roughly 55 tok/s the 35 percent rapid-mlx lead on granite implied.
Why a mixed build alongside: mixed_3_6 beat plain 4-bit on GLM perplexity (11.677 vs 14.457).
Why deleted: the 2026-09-09 sweep ("mixed36 is better and smaller"); Alan: "ok remove the granite extras and the superseded models". Removed 14:08:37Z.
Published? Run-log rows only; no site write-up.

What went wrong

Label as wire id. The first benchmark pass named the server model by the --model-id label instead of the path; all 96 cases (contract 36 + screening 60) per runtime were excluded HTTP 404. Cost: ~1.5 h of queue; fixed by re-running with the model path as id.
One results path, two runtimes. This is exactly the failure a lab rule warns of: same weights on two runtimes are two measurements.
Stale mlx at conversion. The 2026-09-08 conversions used mlx-venv, older than the mlx-main-venv/0.32 used for benchmarking.

What happened to it

Converted 2026-09-08 ~20:00Z → benched 20:07Z to 20:42Z → superseded in conversation by the mixed build → deleted 2026-09-09 14:08:37Z (32 GB). No RESTORE-deleted-*.sh entry for the build; re-creatable from the source with the command above (Qwen3-Next-80B-A3B-Instruct REAP safetensors source (SamsungSAILMontreal) has a RESTORE entry).