MLX 8-bit

A ready-made copy for Apple's MLX. Three tests and a speed trial; deleted in October. Lmstudio-community.

Size: 8.7 GB
Status: Deleted 11 Oct 2026
Scored runs: 3
Smallest Mac: 16 GB estimate, 8K context

Results3 scored · 95% intervals

Suite	Score	95% interval	Passed	Runtime · profile	Recorded
groundingqwen3-8b-mlx-8bitunenforced	74.4%	66.1–81.2	Passed 93/125	mlx_lmextraction	2026-09-08
nullschemaqwen3-8b-mlx-8bitunenforced	56.9%	45.4–67.7	Passed 41/72	mlx_lmextraction	2026-09-08
screeningqwen3-8b-mlx-8bit	96.7%	88.6–99.1	Passed 58/60	mlx_lmextraction	2026-09-08

Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.

The recordHow it was made · how it was served · what went wrong · what happened to it

Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.

File	`models/qwen3-8b-mlx-8bit/` (2 shards)
Format	MLX 8-bit, 8,703,048,738 B (HF)
Where from	downloaded ~2026-08-13 11:20Z
Status	deleted 2026-10-11 (Tier B)
Hugging Face	https://huggingface.co/lmstudio-community/Qwen3-8B-MLX-8bit

Where it came from

Repo: lmstudio-community/Qwen3-8B-MLX-8bit; HF cache ref fea44fa66d05dd6b86ed0e1b382369b2c7464fd3 (.cache/huggingface/hub/models--lmstudio-community--Qwen3-8B-MLX-8bit/refs/main, left behind after the weights were removed; read 2026-10-23).
Date: the HF cache directory was created 2026-08-13 12:20 BST = 11:20Z. The model was in models/qwen3-8b-mlx-8bit/ (2 safetensors shards; 8.1 GiB on the 2026-09-07 listing, shard 1 4.96 GiB). Command not on record (pre-transcript).
Size: 8,703,048,738 B from the current HF repo listing; the local 8.1 GiB matches.
Why: MLX counterpart for the runtime study; eval/dspark_ab.py (mtime 2026-08-13 12:52 BST) points at this directory, 32 minutes after the cache directory appeared.

How it was served

label	dates (UTC)	binary and version	ctx / -np / KV / -fa / `--jinja` / `--reasoning-format`	harness profile	why this label exists
`qwen3-8b-mlx-8bit`	2026-09-07 23:34-23:38 (run-log local 2026-09-08 00:36:05-00:38:48)	`mlx-venv/bin/python -m mlx_lm server`, mlx_lm 0.31.3 ("pinned 0.31.3 for comparability"); run-log `runtime_source` is `note`, not captured from the server	not applicable to mlx_lm; defaults	`extraction`	overnight "never measured" MLX coverage chain; not part of a runtime comparison (the GGUF pair for this model had different weights). Harness invoked with `--model-id models/qwen3-8b-mlx-8bit` and no `--label`, so the label was derived from the path and later normalised to `qwen3-8b-mlx-8bit`

Command (this model at "00:34" BST on the log):

$PYM -m mlx_lm server --model models/qwen3-8b-mlx-8bit
python harness/runner.py --suite {screening,grounding,nullschema} --model <server> \
       --model-id models/qwen3-8b-mlx-8bit --profile extraction --no-resume

Throughput probe in the same chain: 31.1 tok/s (300 tokens in 9.7 s, client-side including prefill).

Other use: eval/dspark_ab.py (DSpark vs baseline via mlx-dspark, dspark/.venv/bin/mlx-dspark, 400 new tokens, 3 prompts x 2 trials), 2026-08-13: baseline 38.0 tok/s mean, DSpark auto-cap 27.3, cap 4 26.4, lookup 28.7 (eval/results_dspark_ab.json).

Decisions

Benched as one of four MLX models the assistant said had never been measured before Alan went to bed ("keep going until everything is done").
Flagged, then spared, on 2026-09-07: the assistant grouped it with the MLX cluster that is "unusable without Xcode and SwiftLM"; the "zero regret" first pass that Alan approved left the MLX cluster alone. Deleted 2026-10-11 in the Tier B sweep (Alan: "nuke tiers a & b", quoted in a lab note), "benched... superseded by the live Q4_0 GGUF preset"; RESTORE-deleted-20261011-tiersAB.sh:59 is a comment-only entry with no download command.
Not published.

What went wrong

Schema not enforced (found 2026-09-08 05:57Z). For granite on the same chain the nullschema arms came back identical (24/24, 24/24, 24/24), and a grep showed response_format and json_schema appear 0 times in mlx_lm's 1,904-line server.py. So this label's nullschema (56.9%) and grounding (74.4%) were free generation. The assistant added a probe-and-refuse guard to harness/runner.py; the first live test of the guard raised AttributeError: 'HTTPBackend' object has no attribute 'honours_schema', whether it was fixed in the same sitting is not on record. The rows were kept and flagged by schema_enforced: false.
Label derived from path. The result files are named ...__qwen3-8b-mlx-8bit__extraction.jsonl; the run-log label qwen3-8b-mlx-8bit was assigned afterwards.
Published-data mismatch: the run-log runtime evidence is a note, not a server capture; weights file bytes come from the current HF repo, not from the deleted directory.
Cross-format comparison trap. Its nullschema (56.9%) sits above the Q4_K_M's (34.7%) but differs in weights, runtime and enforcement, so it is not a format result.
Standing checks: no partial runs; run-log llama_build empty (not llama.cpp); no --jinja concept.

What happened to it

Downloaded ~2026-08-13 -> DSpark A/B 2026-08-13 -> benched 2026-09-07/08 -> listed in the unusable MLX cluster 2026-09-07 -> still on disk 2026-09-16 -> deleted 2026-10-11 (Tier B; comment-only RESTORE entry) -> re-downloadable from lmstudio-community/Qwen3-8B-MLX-8bit (revision may differ).