FreshContextA context lab · Fife
Drawing
02 · Benchmarks
Sheet
2 of 3
Rev
A · 2026-10-01

Qwen3-Next-80B-A3B-Instruct REAP · file← Qwen3-Next-80B-A3B-Instruct REAP

IQ4_XS

mradermacher's GGUF, the first we tested. It matched the full model; deleted the next day, then missed. Mradermacher GGUF.

Size
32.7 GB
Status
Deleted 9 Sep 2026
Scored runs
6
Smallest Mac
64 GB estimate, 8K context
Results6 scored · 95% intervals
SuiteScore95% interval PassedRuntime · profileRecorded
chainwalkqwen3-next-80b-reap-ag50.0%9.5–90.5Passed 1/2llama.cppagentic-fast2026-09-08
groundingqwen3-next-80b-reap87.2%80.2–92.0Passed 109/125llama.cppextraction2026-09-08
nullschemaqwen3-next-80b-reap97.2%90.4–99.2Passed 70/72llama.cppextraction2026-09-08
screeningqwen3-next-80b-reap98.3%91.1–99.7Passed 59/60llama.cppextraction2026-09-08
toolsqwen3-next-80b-reap-ag100.0%79.6–100.0Passed 15/15llama.cppagentic-fast2026-09-08
toolscaleqwen3-next-80b-reap-ag90.0%76.9–96.0Passed 36/40llama.cppagentic-fast2026-09-08

Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.

The recordHow it was made · how it was served · what went wrong · what happened to it

Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.

FileQwen3-Next-80B-A3B-Instruct-REAP.IQ4_XS.gguf (32,664,959,808 B)
FormatGGUF IQ4_XS
Where fromdownloaded 2026-09-08 from mradermacher/...-REAP-GGUF
Statusdeleted 2026-09-09 14:08Z
Hugging Facehttps://huggingface.co/mradermacher/Qwen3-Next-80B-A3B-Instruct-REAP-GGUF

Where it came from

  • Repo and file: mradermacher/Qwen3-Next-80B-A3B-Instruct-REAP-GGUF / Qwen3-Next-80B-A3B-Instruct-REAP.IQ4_XS.gguf. Size 32,664,959,808 bytes (30.4 GiB). sha256 and revision: not on record. The -i1-GGUF imatrix repo was also priced; the plain repo was taken.
  • Command: a one-off script, hf download mradermacher/Qwen3-Next-80B-A3B-Instruct-REAP-GGUF Qwen3-Next-80B-A3B-Instruct-REAP.IQ4_XS.gguf --local-dir qwen3-next-80b-reap with HF_XET_HIGH_PERFORMANCE=1, queued 2026-09-08 09:59:52Z beside an MLX Muse-Glimmer pull.
  • How it went: the slowest download of the programme. The assistant's account at 15:04Z: Wi-Fi on the wrong access point for hours, xet wedged silently, then HF throttled the single stream to 8 Mbps; a rewritten range-request downloader (a one-off script, later eval/tools/pardl.sh) finished it. At 15:03Z the parts totalled 21.6 of 30.4 GB; complete between 15:11Z and the benchmark launch at 15:49Z. Alan, 15:04Z: "man this is slowest dl ever".
  • Who asked: the assistant proposed the test; Alan's "try reap" came at 15:45Z after the file was already landing.
  • Local directory: models/qwen3-next-80b-reap/, reported as 35 GB by du on 2026-09-09, more than the 30.4 GiB file; what else was in it is not on record.

How it was served

labeldates (UTC)binary and versionctx / -np / KV / -fa / --jinja / --reasoning-formatharness profilewhy this label exists
qwen3-next-80b-reap (extraction suites)2026-09-08 15:50:05Z screening, 15:51:48Z grounding, 15:52:36Z nullschema (run-log finish 16:50 to 16:52 BST)llama.cpp/build/bin/llama-server, b10618 eb25b7263-c 16384 -ngl 999 --jinja --alias qwen3-next-80b-reap; -np, KV type, -fa, --reasoning-format left at defaultsextractionthe like-for-like prune test against the full model's 91.7 / 88.8 / 97.2
qwen3-next-80b-reap-ag2026-09-08 17:59:30Z tools, 18:03:01Z toolscale, 18:10:55Z chainwalk (finish 18:59 to 19:10 BST)same-c 32768 -ngl 999 --jinja --alias qwen3-next-80b-reap-agagentic-fastthe full model also held the agentic seat (93.3 / 95.0 / 50.0); REAP had not been tested there, and the Instruct GGUF was not to be deleted first
qwen3-next-80b-reap (contract/2 row)2026-09-08 20:50:54Z (finish 21:50:54 BST)same-c 16384 -ngl 999 --jinja --alias reapextractioncontract/2 head to head. All 48 cases HTTP 503, coverage 0 (see Incidents)
qwen3-next-80b-reap (toolcall/1 row, void)2026-09-09 11:05:48Z (finish 12:05:48 BST)oMLX 0.6.4n/aextractionnot this file. The label was reused for the oMLX pool alias, which pointed at qwen3-next-80b-reap-mlx-mixed36; the run is voided in the run log

Seats held: none. The assistant called it "a seat change waiting to be made" on 2026-09-08, but the router was never repointed at this file; the MLX builds took the "accuracy seat" role next.

No fleet-models.ini preset points at this file: the 2026-09-09 14:08Z reference check found only labs-control.sh and the dsh patch naming the mixed36 build.

Decisions

  • Why fetched: to test whether expert pruning costs the extraction seat its abstention (family doc, Why it was tested).
  • Why kept as the reference: best-scoring REAP build on the board; the extraction and agentic suites showed no loss. Result summary at 15:53Z: a quarter of the parameters gone, 10 GB smaller, 10 percent faster, and "Not approximately: 70 of 72, the same two failures".
  • Why deleted: Alan, 2026-09-09 14:07Z, "ok remove the granite extras and the superseded models". The assistant's sweep listed this directory as "the MLX build supersedes it" and removed it at 14:08:37Z with qwen3-next-80b-reap-mlx-q4. That decision cost a day later: the GGUF was the only REAP build on the 97.2 / 100 side of the record, and on 2026-09-10 Alan had to ask for it back: "I think we deleted the guff once we realised how quick it was in mlx and we settled it, look through notes".

What went wrong

  • contract/2 coverage 0 (HTTP 503 x48). The 20:49Z head-to-head polled /v1/models for readiness, which llama.cpp answers while still loading, so every request hit a loading server. Cost: no contract/2 number for this file; the old 36-row contract/1 file survives as .superseded.
  • Four wrong stories on 2026-09-10. Asked why a re-run read 47.2 nullschema against 97.2 on record, the assistant said in turn: runtime artefact, void run (it had read this file's server as an mlx_lm Qwen3.8-27b server), a mislabelled base-model result, and that this 97.2 was REAP at uniform 4-bit on MLX. The record settles it: that server on 2026-09-08 was this IQ4_XS GGUF under llama-server; the 95.8 was the unpruned mlx-community 4-bit on another server; and mixed36's 47.2 was a different build. Alan's own recollection, 13:07Z: "we measured reap on mlx too though".
  • Standing checks. Partial runs: none (125/60/72 rows; ag runs complete). Stale build stamp: none; b10618 matches the 2026-08-25 rebuild. Missing flags: no --reasoning-format, -fa, KV type; --jinja present.

What happened to it

Downloaded 2026-09-08 ~09:59Z to ~15:30Z → benched 15:49Z to 18:10Z (extraction, agentic) and 20:50Z (contract/2) → listed as superseded 2026-09-09 14:03Z → deleted 2026-09-09 14:08:37Z with qwen3-next-80b-reap-mlx-q4 and the GLM/granite extras. No RESTORE-deleted-*.sh entry (RESTORE-deleted-20260909.sh names the Instruct GGUF, the 180B and the 27B GGUF only). Re-downloadable from the same mradermacher repo; the assistant found it again in an unauthenticated HF search and re-fetched a different quant (Qwen3-Next-80B-A3B-Instruct REAP Q4_K_M (mradermacher GGUF)) the next day. Results survive in the run log.