FreshContextA context lab · Fife
Drawing
02 · Benchmarks
Sheet
2 of 3
Rev
A · 2026-10-01

Model← Models

Qwen3.6 35B-A3B

35B mixture of experts, 3B active, MLX 4-bit (also held as UD-Q5_K_M GGUF), 19.0 GB.

Parameters
35B / 3B active
Quant
MLX 4-bit (also held as UD-Q5_K_M GGUF)
Size
19.0 GB
Family
qwen3.x
RecordStatus · runtime · seat

qwen3.6-35b

Measured, not seatedMoEresidentllama.cppmlxomlx

Model card ↗

Outcome as of 20 September 2026, from the seat registry. The write-up below was made when the model was tested.

Why it was tested

Tested as the fast general MoE, and used as the daily coding driver in DSH. Moved to an MLX build on 2026-09-09.

What came of it

Held the speed record until 2026-09-15, when bonsai-8b-1bit measured 68.5 tok/s on a quiet box. Was: 66.4 tok/s single-stream and 193.2 aggregate at n=32, in 19GB, beating the GGUF build's 52.4. Its weakness is tool-seeking. It calls an announced tool 2/12 and an unmentioned one 0/12, so it needs if-then rules in tool descriptions to be usable agentically, and it goes to 12/12 when it gets them. Requires --tool-call-parser qwen3_coder_xml or its XML tool calls never become OpenAI tool_calls and it looks like it is ignoring every tool. In practice it drives opencode well: 75 tool calls, zero errors, because coding tools are ones the model genuinely cannot answer without.

Recorded runs3 scored · 95% intervals
SuiteScore95% interval PassedRuntime · profileRecorded
grounding82.4%74.8–88.1Passed 103/125llama.cppextraction2026-08-27
nullschema61.1%49.6–71.5Passed 44/72llama.cppextraction2026-08-27
screening86.7%75.8–93.1Passed 52/60llama.cppextraction2026-08-27

Runs marked unenforced were measured through mlx_lm on a suite that requires constrained decoding. mlx_lm accepts the schema and ignores it, so those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.

Every run log entry for this model (5)
FinishedSuiteRuntime · profileScoreCoverageGrader · dataset · config
toolcalltoolcall/1not recordedextraction65.0%Coverage 60/60g 804e8fc9457649d8d 10af88c170513155c 8e5c69081ab588b0
nullschemanullschema/1llama.cppextraction61.1%Coverage 72/72g d7448e0f5fc3f9afd 3b91280bb40372efc 7a652a6a84f42ef5
screeningscreening/1llama.cppextraction86.7%Coverage 60/60g 0661b49a10be2d2ad 64c97ca2269c29afc 4246ff56e771b097
groundinggrounding/1llama.cppextraction82.4%Coverage 125/125g 30b933dcc6a7e2d4d dcb396faf8b83b0ac 279f3a4bd1ff0c7e
groundinggrounding/1llama.cppextraction86.3%Coverage 73/73g 30b933dcc6a7e2d4d dcb396faf8b83b0ac 279f3a4bd1ff0c7e
FamilyOthers tested from the qwen3.x family
  • Qwen3 Coder 30BMeasured, not seated

    30B · MoE · Q4_K_M · 18.0 GB

    The first coding model measured against a real coding contract rather than an extraction suite.

  • Qwen3.5 122B-A10BRemoved

    122B · MoE · IQ3 · 45.0 GB

    Tested as the largest MoE that would fit, on the assumption that more parameters would win.

  • Qwen3.8 27BSeated · agentic

    27B · dense · MLX 4-bit (MTP build, template patched to reasoning_effort=medium) · 15.0 GB

    The dense counterweight to the MoE fleet, and the model where MLX first beat llama.cpp. Promoted to the agent seat on 2026-09-09 after toolcall/1.

  • Qwen3.8-Flash-NextRemoved

    180B · MoE · AD-3.84bpw M64 · 79.1 GB

    Tested because the M64 shard layout claims to run a 180B model on 64GB by keeping a 51B n-gram embedding table on SSD.