FreshContextA context lab · Fife
Drawing
02 · Benchmarks
Sheet
2 of 3
Rev
A · 2026-10-01

Model← Models

Qwen3.8-Flash-Next

180B mixture of experts, 6B active, AD-3.84bpw M64, 79.1 GB.

Parameters
180B / 6B active
Quant
AD-3.84bpw M64
Size
79.1 GB
Family
qwen3.x
RecordStatus · runtime · seat

qwen38-flash-next

RemovedMoEdeletedllama.cpp

Model card ↗

Outcome as of 20 September 2026, from the seat registry. The write-up below was made when the model was tested.

Why it was tested

Tested because the M64 shard layout claims to run a 180B model on 64GB by keeping a 51B n-gram embedding table on SSD.

What came of it

Deleted 2026-09-09 (88 GB). Lost to REAP-60b on every suite where both were measured: screening 95.0 vs 98.3, grounding 81.6 vs 87.2, nullschema 58.3 vs 97.2, chainwalk 0/2 (reached steps 5 and 13 of 200). nullschema is abstention, so 58.3 means it fabricates on ~40% of absent-value cases - the worst weakness for accountancy extraction. It answered the question it was downloaded for (does 40s expert streaming buy better task completion?) and the answer was no. Restore notes in ~/models/RESTORE-deleted-20260909.sh. One reason to revisit: oMLX ships mlx_vlm_qwen4_exp_compat, so it may have an MLX path mlx_lm cannot offer.

Recorded runs3 scored · 95% intervals
SuiteScore95% interval PassedRuntime · profileRecorded
grounding79.2%71.3–85.4Passed 99/125llama.cppextraction2026-09-05
nullschema50.0%38.7–61.3Passed 36/72llama.cppextraction2026-09-05
screening96.7%88.6–99.1Passed 58/60llama.cppextraction2026-09-05

Runs marked unenforced were measured through mlx_lm on a suite that requires constrained decoding. mlx_lm accepts the schema and ignores it, so those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.

Every run log entry for this model (3)
FinishedSuiteRuntime · profileScoreCoverageGrader · dataset · config
nullschemanullschema/1llama.cppextraction50.0%Coverage 72/72g d7448e0f5fc3f9afd 3b91280bb40372efc d248e89da830ce31
screeningscreening/1llama.cppextraction96.7%Coverage 60/60g 0661b49a10be2d2ad 64c97ca2269c29afc dd9b4d06d5abee09
groundinggrounding/1llama.cppextraction79.2%Coverage 125/125g 30b933dcc6a7e2d4d dcb396faf8b83b0ac 9148fcdc2bc14fe4
FamilyOthers tested from the qwen3.x family
  • Qwen3 Coder 30BMeasured, not seated

    30B · MoE · Q4_K_M · 18.0 GB

    The first coding model measured against a real coding contract rather than an extraction suite.

  • Qwen3.5 122B-A10BRemoved

    122B · MoE · IQ3 · 45.0 GB

    Tested as the largest MoE that would fit, on the assumption that more parameters would win.

  • Qwen3.6 35B-A3BMeasured, not seated

    35B · MoE · MLX 4-bit (also held as UD-Q5_K_M GGUF) · 19.0 GB

    Tested as the fast general MoE, and used as the daily coding driver in DSH. Moved to an MLX build on 2026-09-09.

  • Qwen3.8 27BSeated · agentic

    27B · dense · MLX 4-bit (MTP build, template patched to reasoning_effort=medium) · 15.0 GB

    The dense counterweight to the MoE fleet, and the model where MLX first beat llama.cpp. Promoted to the agent seat on 2026-09-09 after toolcall/1.