Model family← Models

Qwen3.5-4B

Qwen's small 4-billion-parameter model. We ran it as the control for NeoHorse-1-4B, a version retrained for agent work, and the original came out ahead overall. It is also a good small tagger.

Files tested: 2
Scored runs: 7

OverviewWhy we tested it · what happened · should you use it

How this page was made. The tests run automatically on our Mac Studio, and every score here comes straight from those runs. This overview was written by Claude, an AI model, from the lab's own records: the run log, notes kept at the time, and the working sessions where each decision was made. Quotes are Alan's words, with spelling fixed. How these pages are made.

Why we tested it

Two reasons, a month apart.

16 August, a speed claim. MTPLX, a tool for Apple's MLX, uses an extra prediction head to draft tokens ahead, and its makers reported 1.6 times the speed on an M4 Mac mini and 2.24 times on an M5 Max. We tried its own Qwen3.5-4B build on the Mac Studio, an older M1 Max, to see if that held.
13 September, as a control. Alan brought in NeoHorse-1-4B, a version of this model retrained for agent work. We ran the original beside it, in the same file format with the same settings and tests, so any difference would come from the retraining alone.

What it took

Nothing unusual: llama.cpp ran unsloth's 4-bit file as it came. In our tables its runs are labelled qwen3.5-4b-base, meaning NeoHorse's base; Qwen's separate pre-trained Base release was never held here.

The speed test needed a second try, alternating normal and drafted runs, because load on the Mac had swung identical runs from 5.6 to 14.7 tokens a second.

What happened

The original beat its retrained version overall: 461 of 625 test cases against NeoHorse's 432. NeoHorse reached for tools more often, 56.7% against 36.7% on our tool test. It lost on finding and citing answers in a document (63.2% against 74.4%), on tagging (86.2% against 93.8%) and on answering from the document it was given rather than its own knowledge (75.0% against 85.4%). Its makers report it ahead of this model on their own ten benchmarks.
It is a good small tagger. 93.8% on our tagging test with empty fields allowed, in a 2.6 GB file, above the 90.4% of the small Gemma we used as the bar for its size. We never followed it up.
It is weak at saying "I don't know". It declined only 33.3% of the time when the answer wasn't there.
Drafting ahead made it slower on the Mac Studio: 0.954 times normal speed drafting one token ahead, 0.805 times drafting two. The build's own notes say some early builds had a faulty drafting head that made them slower, and we can't tell whether ours was one, so this may not be the last word. When Alan raised MTPLX again in September, "the other runtime we haven't tested yet is mtplx", we left this model out: a 4B reads so little per token that drafting has little to save.

The 4-bit file was deleted on 11 October in a clear-out of models tested once.

What it taught us

Test a retrained model against its own original. Same file format, same settings: the difference is the retraining. Here it bought tool use and cost accuracy.
A speed-up measured on newer Macs may not survive on older ones. Speculative decoding does not pay on this box except embedded MTP on llama.cpp.

Should you use it?

For tagging on a small Mac, it is a cheap, capable choice. Not where it must say "I don't know", and not for tool use: it reaches for a tool only about a third of the time it should. For small-model tool calling, K2-Horizon-3.7B called the tool 60 times in 60 here. Don't count on MTPLX's speed-up on an M1-generation Mac: we didn't see it on ours. Check the smallest Mac for the 4-bit file below.

The files2 tested · each has its own page

MLX 4-bit + MTP headkept
An MLX build with an extra head for drafting tokens ahead, for a speed test. Still on disk, in a folder our clear-outs never looked in.
MTPLX Optimized-Speed, Youssofal · On disk · 2.4 GB · never scored
Q4_0
Unsloth's 4-bit build, run beside NeoHorse as the control. Seven tests, then deleted.
Unsloth GGUF · Deleted 11 Oct 2026 · 2.6 GB · 7 scored runs · smallest Mac 8 GB

The lab's recordEvery decision, with its date · the model's specification

Decisions, in order

2026-08-16 — MTPLX A/B (reconstructed from scripts; no transcript). eval/mtplx_ab.py (14:44 local) swept depth 0–3 within one server; its result file is all zeros (see the MTPLX variant's incidents). eval/mtplx_paired.py (15:24 local) replaced it with interleaved AR/MTP pairs because "six consecutive runs of the same 4B ranged 5.6 to 14.7 tok/s purely from GPU contention" (its docstring). Verdict SLOWER at both depths. Who asked is not on record.
2026-09-13 06:50Z — fetch the base as a control (assistant's design). "the base isn't on disk, and the base is what makes this interesting". Four repos were probed for a Q4_0; unsloth/Qwen3.5-4B-GGUF had one. Q4_0 rather than K-quant because of the byte-alignment finding ("the right format per byte-alignment").
2026-09-13 07:24Z — run started late. The assistant had said it would wait for a go-ahead; at Alan's "How's the benchmark coming along?" it admitted "it hadn't started until just now" and that the delay was "on me, not on the box" and launched.
2026-09-13 08:10Z — result: post-training is a trade. The assistant offered a finding write-up ("Want this written up as a finding …?"). No finding was written for it; whether that was declined or dropped is not on record.
2026-10-11 — deleted in the Tier B sweep under Alan's "nuke tiers a & b" (quoted in a lab note; the line itself is not in our session records). Listed as "benched-once, abandoned".

What we learned, in the lab's words

NeoHorse's agentic post-training is a trade against this base, not an improvement. Same quant, runtime, flags and datasets: toolcall +20.0 (McNemar p=0.004) for NeoHorse, but grounding −11.2 (p=0.001), tagger_nullable −7.5 (p=0.001), epistemics −10.4 (p=0.062); toolscale and worldview identical; overall 432/625 vs 461/625 for this base. The vendor reports +5.93 macro over the base on its own ten benchmarks.
The base already has partial tool perception: toolcall 22/60 (answered_directly 38, called 22) — well above qwen3-8b and spark-x2.5-4b at 0/60, below k2-horizon-3.7b at 60/60. Finding Tool perception is binary and tracks the checkpoint, not size or vendor claims was written before this run (2026-09-10) and its "nothing between" wording does not cover a 36.7% model.
It is a strong small tagger: tagger_nullable 225/240 = 93.8% (run log) at 2.4 GB, above gemma4-e2b's 90.4% (217/240) that the assistant used as the 4B-class bar. It was never considered for the TAG seat; not on record why.
MTP speculation does not pay on the M1 Max for this model. The MTPLX build at draft depth 1 ran at a median 0.954× of autoregressive (IQR 0.927–0.979) and at depth 2 at 0.805× (IQR 0.795–0.833), both verdict SLOWER (2026-08-16). Consistent with Speculative decoding does not pay on this box except embedded MTP on llama.cpp.

Why it was tested, in the lab's words

Two separate reasons, a month apart.

2026-08-16 — the MTPLX speed claim (pre-transcript). MTPLX publishes 1.6× on an M4 Mac mini and 2.24× on an M5 Max for native MTP speculation; the question was whether that transfers to an M1 Max, which mtplx hardware inspect reported as hardware_acceleration_eligible: false (docstring of eval/mtplx_ab.py). The subject was MTPLX's own Youssofal/Qwen3.5-4B-MTPLX-Optimized-Speed. No session transcript covers 2026-08-16; the record is the two scripts and their result files.
2026-09-13 — the control arm for NeoHorse-1-4B. Alan, 06:49Z: "We have be new model to put through benchmarks neohorse 1 4b". The assistant found NeoHorse is a post-train of Qwen3.5-4B and fetched the base alongside it: "Same architecture, same size, same quant — only the post-training differs. That's the arm that answers the vendor's actual claim". It was never a seat candidate in its own right; the sweep note calls it "the base-model control arm" (a lab note).

Open questions

Is NeoHorse's base really Qwen/Qwen3.5-4B rather than Qwen3.5-4B-Base? The web summary says "post-training Qwen3.5-4B"; the card itself was not read. The control arm assumes the former.
The label qwen3.5-4b-base names the role (NeoHorse's base), not Qwen's -Base checkpoint. A reader of the site will take it as the pre-trained base model.
Would the post-trained-vs-base trade hold under the vendor's own sampling settings? Both arms ran extraction (temperature 0, thinking off) and study-nothink.
MTPLX v2.7.1 (installed 2026-09-11) was never re-run on this build; the 2026-08-16 MTPLX version is not on record. The README's "July 2026 rebuild" note says earlier sidecars "made the model slower than plain decoding" — whether the held snapshot (forged 2026-07-18) predates or includes that fix is not determinable from the files.
A tagger_nullable of 93.8% at 2.4 GB was never followed up for a utility seat.

The model in detail

field	value
vendor	Qwen (Alibaba)
upstream repo	`Qwen/Qwen3.5-4B` (post-trained model); a separate `Qwen/Qwen3.5-4B-Base` checkpoint also exists and was never held here
revision	not on record (the GGUF was fetched from `resolve/main`)
licence	Apache 2.0 is stated for the derivative NeoHorse; the Qwen3.5-4B licence itself is not on record here
released	2026-02-27 (HF `createdAt`)
architecture (from the held MLX build's config.json, `model_type qwen3_5_text`)	32 layers, of which 8 are full attention (`full_attention_interval 4`) and 24 linear attention; hidden 2560; 16 attention heads, 4 KV heads, head_dim 256; FFN 9216; vocab 248,320; tied embeddings; `max_position_embeddings` 262,144; one MTP layer declared (`mtp_num_hidden_layers 1`)
multimodal	the config is `Qwen3_5ForConditionalGeneration` with image/video preprocessors; no vision path was ever exercised here

Derivatives held as their own families: NeoHorse-1-4B (TokenRhythm's agentic post-train of Qwen3.5-4B).