MLX conversion of zai-org/GLM-4.7-Flash for Apple silicon. At the time of conversion no MLX build of this model existed.
Model
| architecture | glm4_moe_lite, MoE, 64 routed experts + 1 shared, 4 active per token |
| layers / hidden | 47 / 2048 |
| attention | 20 heads, 20 KV heads |
| context | 202,752 |
| base precision | bfloat16, 58.2 GB across 48 shards |
| this build | 8-bit, group size 64, affine, 30 GB |
Conversion
mlx_lm convert --hf-path zai-org/GLM-4.7-Flash \
--mlx-path glm-4.7-flash-mlx-q8 -q --q-bits 8 --q-group-size 64mlx-lm0.32.0 at commit7fb4be44d560e5b74595210f83cb6003a57e52a7, built from git main becauseglm4_moe_liteis not in the 0.31.3 PyPI releasemlxcore 0.32.2, Apple M1 Max, 64 GB, macOS 15
Requires mlx-lm from git main until glm4_moe_lite reaches a release.
Measured
Screening is a second-pass judging task over 60 held-out cases. It is the only suite here that both runtimes can be compared on, for the reason given under Limitations.
| build | size | screening | 95% CI |
|---|---|---|---|
| MLX 8-bit (this) | 30 GB | 96.7% | 89.0 – 99.1 |
| MLX 4-bit | 16 GB | 96.7% | 89.0 – 99.1 |
GGUF Q8_0, llama.cpp | 30 GB | 95.0% | 86.3 – 98.3 |
| MLX mixed 3-6 bit | 14 GB | 93.3% | 84.1 – 97.4 |
| MLX mixed 4-6 bit | 17 GB | 93.3% | 84.1 – 97.4 |
Those intervals overlap. Read this as "no measurable regression from the conversion", not as MLX being better.
Throughput on an M1 Max, 300 tokens, greedy: 21.3 tok/s at 8-bit, 30.2 tok/s at 4-bit.
Context behaviour is good and was initially misjudged by us. Measured on a ladder from 8k to 131k, resident memory went 29.75 GB to 36.42 GB, so 6.67 GB of KV across the whole range, holding 37.2 tok/s throughout. A prediction from its head counts said 72 GB and was wrong by a factor of seven.
Limitations
Weak at declining to answer. On a suite that tests whether a model returns null rather than inventing a value when a field is genuinely absent, the base model scores 48.6% under llama.cpp with schema enforcement active. It fabricates on roughly half of those cases. If you are extracting structured data from documents where fields are often missing, evaluate this carefully. That is a property of the base model, not of this conversion.
4-bit costs more than size. Measured under identical unenforced conditions, dropping from 8-bit to 4-bit cost 23 points of grounding accuracy (76.8 to 53.6) and 19 points of abstention (59.7 to 40.3). The 4-bit build is 40% faster and 14 GB smaller. Choose accordingly.
Two suites cannot be run on MLX at all. mlx_lm's server accepts response_format and ignores it. response_format and json_schema appear zero times in its 1,904-line server.py. Any suite depending on constrained decoding silently measures free generation instead, so those results are excluded here rather than reported.
The mixed-precision builds are not published. They are smaller but scored lower on the only comparable suite, and config.json reports all three 4-bit variants identically, which would mislead anyone reading the metadata.
Private evaluation data. The suites are internal and these numbers are not independently reproducible. Dataset identifiers are content hashes, published so results can be tied to a fixed case set, not so the cases can be recovered.
License
MIT, inherited from the base model.