The measured state of local AI.

Two seats detect temporal conflicts, not one: k2-horizon-36b matches ornith at 4/4

Detecting a conflict where one record simply dates itself later and disagrees is something ornith-35b-a3b and k2-horizon-36b-a4b both do at 4/4. The other three seats score 0-50%. No seat produces false positives, so the fleet's failures are uniformly missed conflicts rather than invented ones.

contradiction/1 probe, 48 cases (32 conflict across 8 shapes, 16 no-conflict controls), identical prompt and grader for every seat. implicit arm: ornith 4/4, bonsai-2 PTQ1_0 2/4, btl-4 1/4, edge0-35b 0/4, k2-horizon-3.7b 0/4. Overall conflict detection: ornith 91%, btl-4 78%, bonsai-2 75%, edge0 72%, k2 12%. Controls: 16/16 for all five. The headerless arm -- a figure in a table row whose column header is absent -- is 0/4 for four seats and 1/4 for ornith, a uniformity that indicates the case rather than the models. Added 2026-09-18: k2-horizon-36b-a4b Q4_0, same probe, implicit 4/4, overall 39/48 (81%), controls 16/16, headerless 0/4 like everyone else. The original version of this finding named ornith as the only capable seat because k2-36b had not been run -- it was tested only in its 3.7B form, which scores 12% on conflict detection and is a tool seat.

Action Route temporal-conflict work to ornith or k2-horizon-36b. Do not generalise from a model family's small variant to its large one: k2-horizon-3.7b at 12% and k2-horizon-36b at 72% conflict detection are the same family and not interchangeable evidence.

SwiftLM allocates ~10x mlx_lm's transient memory per prefill token, which is what created every context ceiling attributed here to models or hardware

Measured with a fresh server per data point and a single request each, SwiftLM's peak process footprint rises ~29-34 GB over idle while prefilling a 12,000-token prompt. mlx_lm rises 3.0 GB on the same prompt, on two different models. The transient is roughly linear in prompt length at ~2.5 GB per 1,000 tokens, which pushes a 20 GB model past a 64 GB machine at around 15K tokens.

Fresh server, one request, peak phys_footprint sampled at 2 Hz. qwen3.6-35B-A3B-4bit at 12,000 tokens: SwiftLM idle 19.0 -> peak 53.0 GB (delta 34.0), mlx_lm idle 19.0 -> peak 22.0 GB (delta 3.0). gemma-4-26b-a4b-4bit at 12,000 tokens: SwiftLM 14.0 -> 43.0 (delta 29.0), mlx_lm 14.0 -> 17.0 (delta 3.0). SwiftLM's own GPU_MEM stays flat at 19-21 GB (qwen) and 15.6-15.8 GB (gemma) across every context depth, so the growth is host-side and not the GPU KV cache -- its own kv_cache_gb estimate is 0.3 GB. mlx_lm continues to 60,000 tokens at 29.0 GB total and dies near 70,000 on '[METAL] Command buffer execution failed: Insufficient Memory'. The mechanism inside SwiftLM is NOT established: the magnitude is ~3x a full-vocabulary fp32 logit vector per position, but qwen has the smaller vocabulary (248,320 vs 262,144) and the larger transient, so vocabulary does not explain it and the prefill loop in their mlx-swift-lm fork was not read.

Action Use mlx_lm for long-context work; treat SwiftLM's context ceiling as a property of SwiftLM until its prefill allocation is understood. Measure memory with a FRESH server per data point: both runtimes retain allocator pools across requests, and reusing one server produced internally contradictory per-token figures here (12,233 KB/token at 3K falling to 1,818 KB/token at 30K on the same server) that are artefacts of pool retention, not measurements of anything.

Abstention ranges 18-61 percent across the fleet, and declining is not the same skill as noticing a correction

CORRECTED 2026-09-15, twice in one day. The original claim -- that nullschema sits between 18 and 44 percent across every model measured, and that BTL-4 Q4_K_M at 44.4 was the first above 40 -- was FACTUALLY WRONG. qwen3.6-35b-a3b scored 61.1 percent (n=72) and was already in the results when the finding was written; it was omitted from the comparison set. The real range is 18.1 to 61.1. Abstention is still a fleet weakness, but it is not a wall at 44, and the gap between best and worst is far wider than stated.

nullschema, 72 cases each: qwen3.6-35b-a3b 61.1, BTL-4 Q4_K_M 44.4, BTL-4 IQ2_XXS 36.1, ornith-35b-a3b-q40 36.1, MiniCPM5-2B 31.9, ternary-bonsai-27b 27.8, bonsai-8b-1bit 19.4, bonsai-27b-1bit 18.1, qwen3.8-9b-distill 18.1. nullschema, 72 cases each: BTL-4 Q4_K_M 44.4, BTL-4 IQ2_XXS 36.1, ornith-35b-a3b-q40 36.1, ternary-bonsai-27b 27.8, bonsai-27b-1bit 18.1, bonsai-8b-1bit 19.4, qwen3.8-9b-distill 18.1. Best-in-class before this session was 36.1. CRITICAL QUALIFIER: worldview/2 breaks abstention into arms, and the ordering inverts. On the `superseded` arm -- a fact whose correction is also on record -- BTL-4 Q4_K_M scores 0/8 and ternary-bonsai-27b 0/8, while ornith-35b-a3b-q40 scores 8/8. BTL-4's failure mode is recorded as right_value_not_flagged (8) and overclaimed (7): it retrieves the stale value and states it without qualification. So the best abstainer on absent fields is among the worst at spotting a superseded one. `single_source` is worse still and fleet-wide: 0/8, 0/8, 2/8 -- nothing reliably flags uncorroborated evidence, consistent with models-do-not-detect-conditions.

Action Build the comparison set from every label in benchmarks/results, not from the models under discussion. This finding was written from a four-model table while nine models had scores. Keep admit.py in front of anything that writes to the graph regardless of which model is serving. Use nullschema as the gate for any new model considered for an extraction role, and treat anything below 50 percent as requiring external grounding. Screen an extraction candidate on nullschema AND on worldview/2's superseded and single_source arms, not nullschema alone. Where a role depends on using the latest version of a record, ornith-35b-a3b is currently the only model measured that does it reliably.

Q4_0 decodes up to 2x faster than Q4_K_M at the same bit-width, and emits valid JSON more reliably

REVISED -- scoped to DENSE models; unvalidated on MoE. Q4_0 and Q4_K_M are both ~4.5 BPW. Q4_0 decodes 1.25-2.04x faster and produces parseable output under constrained decoding far more often. It does NOT improve abstention judgement -- that was a formatting artefact, corrected below.

Identical weights, one variable. granite-guardian-8b 36.84 -> 52.22 tok/s; Qwen3.5-9B 31.24 -> 43.84 at tg64 and 21.60 -> 44.11 at tg128; qwen3.8-27b 9.79 -> 15.18; k2-36b 39.85 -> 49.67. Q4_K_M DEGRADES with output length while Q4_0 holds flat, so the gain is larger in real use than short benchmarks show (1.40x at 64 tokens, 2.04x at 128). On k2-36b's nullschema the headline 54.2 -> 65.3 decomposes as: malformed 12 -> 3, misattributed 21 -> 22. The gain is entirely fewer unparseable responses under response_format; the grader marks those enforcement_absent, i.e. arguably void runs. Judgement was unchanged. Mechanism: Q4_K_M uses 256-element super-blocks with 6-bit quantised scales and promotes some tensors to Q6_K; Q4_0 is 32-element blocks with a full fp16 scale, two weights per byte. SCOPE (2026-09-14): every model in this evidence set whose architecture is recorded is DENSE -- granite-guardian-4.1-8b and qwen3.8-27b confirmed from their GGUF headers; k2-36b is no longer on disk and its architecture is not recorded anywhere. The ratio above is therefore a dense-model result and was stated without scope. Suggestive counter-observation, NOT a measurement: two 35B-A3B MoEs serving side by side gave agents-a1 (Q4_K_M) 49.1 t/s against ornith-1.5 (Q4_0) 53.4 t/s, a ratio of 1.10x. Those are DIFFERENT WEIGHTS, so it is not a quantisation comparison and cannot confirm anything. No MoE model exists on this disk in two formats, so the clean same-weights test has not been run.

Action Do NOT apply this ratio when sizing or selecting an MoE seat -- it is unvalidated there. To close the gap, requantise one MoE to a second format (agents-a1 Q4_K_M -> Q4_0 is the cheapest path; decode speed depends on layout, not on requantisation quality loss) and run llama-bench on both. RESOLVED 2026-09-15: the MoE question is now measured, and the answer differs by format family -- see iq-codebook-penalty-transfers-to-moe-kquant-does-not. K-quant penalty falls to 1.10x on a sparse MoE; IQ codebook penalty RISES to ~2.1x. Use that finding, not this one, when sizing an MoE seat.

Byte alignment, not bit count, governs decode speed

REVISED -- scoped to DENSE models; unvalidated on MoE. Quantisations whose bit width divides a byte evenly (8, 4, 2) sustain far more of peak memory bandwidth than those that straddle byte boundaries (5, 6). A smaller file can be slower than a larger one.

Qwen3.5-9B, same weights requantised four ways, repeated on two prompts with under 0.3 percent variance. Q4_0 45.4 tok/s at 60 percent of 400 GB/s; Q6_K 32.1 at 59 percent; Q8_0 31.4 at 75 percent; Q5_0 30.5 at 48 percent, the worst despite being smaller than Q6_K and Q8_0. Stock Q4_K_M 32.8 at 47 percent. Independently reproduced by llama-bench: Q4_0 44.59 +/- 0.01 against Q4_K_M 31.45 +/- 0.03, a 1.42x gap. Same pattern on the 27B: Q4_0 15.8 tok/s at 62 percent beats IQ3_S 11.5 at 35 percent while being 30 percent larger. SCOPE (2026-09-14): every model in this evidence set whose architecture is recorded is DENSE -- granite-guardian-4.1-8b and qwen3.8-27b confirmed from their GGUF headers; k2-36b is no longer on disk and its architecture is not recorded anywhere. The ratio above is therefore a dense-model result and was stated without scope. Suggestive counter-observation, NOT a measurement: two 35B-A3B MoEs serving side by side gave agents-a1 (Q4_K_M) 49.1 t/s against ornith-1.5 (Q4_0) 53.4 t/s, a ratio of 1.10x. Those are DIFFERENT WEIGHTS, so it is not a quantisation comparison and cannot confirm anything. No MoE model exists on this disk in two formats, so the clean same-weights test has not been run.

Action Serve Q4_0 wherever a build exists. Do not chase sub-4-bit for speed; the unpacking cost grows faster than the byte saving. Do NOT apply this ratio when sizing or selecting an MoE seat -- it is unvalidated there. To close the gap, requantise one MoE to a second format (agents-a1 Q4_K_M -> Q4_0 is the cheapest path; decode speed depends on layout, not on requantisation quality loss) and run llama-bench on both. RESOLVED 2026-09-15: the MoE question is now measured, and the answer differs by format family -- see iq-codebook-penalty-transfers-to-moe-kquant-does-not. K-quant penalty falls to 1.10x on a sparse MoE; IQ codebook penalty RISES to ~2.1x. Use that finding, not this one, when sizing an MoE seat.

Quantisation format changes output CONVENTIONS, not abstention judgement

REVISED. The original claim -- that Q4_0 helps from uniform K-quants and hurts from dynamic quants -- read two different formatting artefacts as opposite capability changes. The measure of actual fabrication is unchanged by format in both directions.

Decomposing nullschema failures by type: k2-36b Q4_K_M -> Q4_0 moved malformed 12 -> 3 with misattributed 21 -> 22. qwen3.8-27b UD -> Q4_0 moved verbal-declines 6 -> 11 with misattributed 11 -> 11, identical. So k2 'improved' by emitting valid JSON more often and the 27B 'degraded' by phrasing refusals as text more often. Neither model changed how often it asserted a wrong value. A third model, Qwen3.5-9B, appeared to lose 16.7 points; 24 of 24 absent-case failures were the literal phrase 'not mentioned' -- a correct refusal in an unaccepted form.

Badly-chosen mixed precision costs abstention; sensitivity-aware mixing protects it

REVISED 2026-09-11. The original claim -- that non-uniform precision degrades abstention at equal average bit-width -- fails on both halves. The bit-widths were never equal, and a principled mixed-precision scheme BEATS uniform quantisation on abstention. What matters is whether the allocation is sensitivity-aware, not whether it is uniform.

The original comparison used qwen3-next-80b-reap mlx-mixed36, which declares bits:4 but overrides 487 of 542 tensors DOWN to 3-bit -- a bit-width comparison wearing a mixed-precision label. Reconverting the same prune at mixed36's own base settings (4-bit, group 64, affine, no overrides) gave nullschema 84.7% vs 47.2%, but toolcall fell 78.3% -> 25.0%. Decisively, on qwen3.8-27b the direction reverses: its Unsloth Dynamic baseline (per-tensor overrides protecting sensitive tensors) scores nullschema 76.4%, and uniform Q4_0 on the same weights scores 69.4% -- 0 cases fixed, 5 broken, every one an abstention case (3 lure, 2 absent). Mixed precision chosen by sensitivity protects exactly the capability uniform quantisation loses.