FreshContextA context lab · Fife
Drawing
02 · Benchmarks
Sheet
2 of 3
Rev
A · 2026-10-01

Qwen3.6-35B-A3B · file← Qwen3.6-35B-A3B

MLX 4-bit

A ready-made copy for Apple's MLX. Most of the tool-seeking tests ran on it; Alan's coding model on 8 and 9 September. Mlx-community.

Size
20.4 GB
Status
On disk
Scored runs
4
Smallest Mac
36 GB estimate, 8K context
Results4 scored · 95% intervals
SuiteScore95% interval PassedRuntime · profileRecorded
contractmlx-community-Qwen3.6-35B-A3B-4bit27.1%16.6–41.0Passed 13/48rapid-mlxextraction2026-09-08
screeningmlx-community-Qwen3.6-35B-A3B-4bit90.0%79.9–95.3Passed 54/60rapid-mlxextraction2026-09-08
screeningqwen3.6-35b-swiftlm88.3%77.8–94.2Passed 53/60swiftlmextraction2026-09-17
toolcallmlx-community-Qwen3.6-35B-A3B-4bit60.0%47.4–71.4Passed 36/60rapid-mlxextraction2026-09-09

Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.

The recordHow it was made · how it was served · what went wrong · what happened to it

Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.

FileHF cache models--mlx-community--Qwen3.6-35B-A3B-4bit @ 38740b84
FormatMLX 4-bit affine g64, router gates 8-bit; 4 safetensors shards, 20,402,204,271 B
Where fromdownloaded 2026-09-08 by rapid-mlx pull
Statuson disk; served 2026-09-08 23:44Z to 2026-09-09 16:28Z; benched
Hugging Facehttps://huggingface.co/mlx-community/Qwen3.6-35B-A3B-4bit

Where it came from

  • Repo: mlx-community/Qwen3.6-35B-A3B-4bit, snapshot 38740b847e4cb78f352aba30aa41c76e08e6eb46 (refs/main in the HF cache, read 2026-10-23).
  • Command: HF_XET_HIGH_PERFORMANCE=1 rapid-mlx pull mlx-community/Qwen3.6-35B-A3B-4bit inside a one-off script. Pull progress showed 20,429,169,263 bytes total for the repo.
  • Date: 2026-09-08, shards written 22:55–22:58 BST (21:55–21:58Z) (snapshot mtimes).
  • Size: 4 safetensors shards, 20,402,204,271 B (stat 2026-10-23). sha256 not on record (blob names are the LFS oids, e.g. 09f3e6ec… for shard 1).
  • Format: qwen3_5_moe, 4-bit affine, group 64; every layer's mlp.gate and mlp.shared_expert_gate at 8-bit (config.json quantization). Includes the vision tower config.
  • Who / why: Alan's "ok, one model I want to revisit 3.6 moe model in mlx / rapid mlx"; rapid-mlx's own recipe named it the "Fast" pick for a 64 GB Mac.

How it was served

labeldates (UTC)port / runtimeflags that matteredharness profilewhy this label exists
mlx-community-Qwen3.6-35B-A3B-4bit (screening)2026-09-08 ~22:01Zrapid-mlx 0.13.4, temporarily in the router's placerapid-mlx serve <repo> --max-num-seqs 32 --max-concurrent-requests 32 --gc-control; no tool parserextractionfirst battery: scenarios, concurrency, contract, screening
mlx-community-Qwen3.6-35B-A3B-4bit (contract)2026-09-08 22:05Zsame--enable-auto-tool-choice --tool-call-parser qwen3_coder_xml added; pre-flight on the exact model idextractionre-run after the no-parser hypothesis; same 27.1
q36-toolparser2026-09-08 22:03Zsame, with the parser--model-id "q36-toolparser" — 404 on all 48 casesextractiona mistaken model id; measured nothing
mlx-community-Qwen3.6-35B-A3B-4bit (toolcall)2026-09-08 ~23:03Zsame--gc-control --enable-auto-tool-choice --tool-call-parser qwen3_coder_xmlextractionfrozen toolcall/1 across three models
qwen3.6-35b (toolcall, void)2026-09-09 10:50Z–10:58ZoMLX 0.6.4, model dir symlink .omlx/models/qwen3.6-35b → this snapshotomlx serve --model-dir … --paged-ssd-cache-dir … --memory-guard balancedextractiontoolcall/1 "against the DEPLOYED stack"; filed under the GGUF's label because oMLX served the id qwen3.6-35b
qwen3.6-35b-swiftlm2026-09-17 19:01Z–19:04ZSwiftLM release build (commit not recorded)./SwiftLM --model <snapshot> --parallel 1 --temp 0 --max-tokens 2048; coherence gate passedextraction (temperature 0, seed 42 in manifest)capability check that SwiftLM serves this model correctly

Run-log runtimes: rapid-mlx 0.13.4 (mlx 0.32.2, mlx_lm 0.31.3 in the rapid-mlx env), omlx 0.6.4, swiftlm (version null). llama_build stamps on q36-toolparser (b10618), the void row (b10618) and qwen3.6-35b-swiftlm (10893) are backfill artefacts: no llama.cpp served these runs.

Seats held: labs-control.sh as qwen3.6-35b, the opencode default cortex-mlx-36/qwen3.6-35b, from 2026-09-08 23:44Z; moved to oMLX 08:30Z–12:19Z on 2026-09-09; back 12:19Z; stood down 2026-09-09 16:28Z by pkill -f "rapid-mlx serve". Also a "Fast general seat" in the oMLX inventory that morning.

Decisions

  • Downloaded 2026-09-08 for the MLX revisit; registered for opencode/pi at Alan's request (family decision log).
  • Not removed when stopped; still in the HF cache 2026-10-23 (19 GB). It was flagged "never benchmarked" in a 2026-09-11 disk triage table that keyed on models only — it had five run-log rows by then.

What went wrong

  • No tool parser on first serve. contract scored 13/48 with 32 no_json classifications; the parser fix did not move the score, and a grader bug had mis-classified null abstentions (32 of 48 reclassified, no pass/fail change). The first file is kept as contract__…jsonl.superseded.
  • Wrong model id. q36-toolparser 404'd every case; a pre-flight was added. Published data drops it as measured-nothing.
  • Label collision on oMLX. oMLX served this snapshot as qwen3.6-35b, so its toolcall run landed in the GGUF's results file name and label. Voided 2026-09-09 and in published data 2026-10-21. The run log still files it under qwen3.6-35b.
  • Overnight opencode run (2026-09-09). The session ran on this model 00:47–00:52 BST, then switched to qwen3.8-27b at 00:54 and stalled; the "roguelike was a washout" (Alan) belongs to the 27B half (reconstructed).
  • SwiftLM context ceiling published wrongly as a model/hardware limit (~14K), then revised the same night.

What happened to it

Downloaded 2026-09-08 → benched 2026-09-08/09 → served on rapid-mlx / oMLX 2026-09-08 23:44Z – 2026-09-09 16:28Z → SwiftLM test 2026-09-17 → on disk 2026-10-23 (HF cache, 20,402,204,271 B). No RESTORE entry needed. Re-downloadable from mlx-community/Qwen3.6-35B-A3B-4bit @ 38740b84.