Ornith 1.5 35B-A3B · file← Ornith 1.5 35B-A3B
MLX 4-bit
A copy for Apple's MLX, to compare with llama.cpp. Scored once, then deleted.
- Size
- 19.5 GB
- Status
- Deleted 8 Sep 2026
- Scored runs
- 3
- Smallest Mac
- 36 GB estimate, 8K context
| Suite | Score | 95% interval | Passed | Runtime · profile | Recorded |
|---|---|---|---|---|---|
| groundingornith-1.5-35b-mlx-4bitunenforced | 67.2% | 58.6–74.8 | Passed 84/125 | mlx_lmextraction | 2026-09-08 |
| nullschemaornith-1.5-35b-mlx-4bitunenforced | 62.5% | 51.0–72.8 | Passed 45/72 | mlx_lmextraction | 2026-09-08 |
| screeningornith-1.5-35b-mlx-4bit | 88.3% | 77.8–94.2 | Passed 53/60 | mlx_lmextraction | 2026-09-08 |
Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.
Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.
| File | ornith-1.5-35b-mlx-4bit/ (4 shards, 14 files) |
| Format | MLX 4-bit, 19.53 GB |
| Where from | downloaded 2026-08-20 (ornith-mlx-dl.log); repo not in the log |
| Status | deleted 2026-09-08 16:14Z |
| Hugging Face | https://huggingface.co/ornith-ai/Ornith-1.5-35B-A3B-MLX-4bit (size match only) |
Where it came from
- Directory:
models/ornith-1.5-35b-mlx-4bit/, 4 safetensors shards, 14 files, 19.53GB in 1136s = 138 Mbps (models/ornith-mlx-dl.log, mtime 2026-08-20 23:50 local). configmodel_typeqwen3_5_moe, 4-bit. - Revision, sha256: not on record. Date: 2026-08-20.
How it was served
| label | dates (UTC) | binary and version | ctx / -np / KV / -fa / --jinja / --reasoning-format | harness profile | why this label exists |
|---|---|---|---|---|---|
ornith-1.5-35b-mlx-4bit (results files *__ornith-1.5-35b-mlx-4bit__*) | 2026-09-07 ~23:10–23:15Z (run-log ts 2026-09-08 00:11:47–00:15:08 BST) | mlx-venv/bin/python -m mlx_lm server, mlx_lm 0.31.3 ("pinned 0.31.3 for comparability") | server defaults (no context flag; no KV quant) / n/a / n/a / n/a; response_format accepted and ignored | extraction | a runtime comparison against the Q6_K GGUF baseline, run after granite's MLX build; recorded at the time as "a looser comparison... bit depth, not just runtime" |
Command: a one-off script, harness/runner.py --suite {screening,grounding,nullschema} --model-id models/ornith-1.5-35b-mlx-4bit --profile extraction --no-resume. The model id is the path because mlx_lm 404s on any other name. The run-log label was shortened later; results files keep the path label.
Seat: none on the locked harness. On 2026-08-20 it was served by eval/bench_ornith_mlx.py for the prefill comparison.
Decisions
- Downloaded 2026-08-20 to test MLX on an MoE; the conclusion that day was "MLX is not better here, on either architecture" (a lab note).
- Kept unused 2026-08-26 to 2026-09-07 as an MLX duplicate.
- Deleted 2026-09-08 16:14Z to make room for the REAP MLX conversion. Alan: "sure nuke them for now the scored were naff right?".
What went wrong
- Unenforced schema (mlx_lm silently ignores response_format, 2026-09-08): mlx_lm accepts
response_formatand never applies it. This file's nullschema 62.5 was 28 points above its GGUF sibling for that reason, and on 2026-09-12 it was still listed on a board asornith-1.5-35bat 62.5%.
What happened to it
Downloaded 2026-08-20 → prefill probe 2026-08-20 → idle → locked suites 2026-09-07/08 → deleted 2026-09-08 16:14Z. No RESTORE entry. Re-downloadable if the size inference is right from https://huggingface.co/ornith-ai/Ornith-1.5-35B-A3B-MLX-4bit.