FreshContextA context lab · Fife
Drawing
02 · Benchmarks
Sheet
2 of 3
Rev
A · 2026-10-01

Qwen3.8-27B · file← Qwen3.8-27B

Splash package

A package for the Splash runtime. About twice as fast on one conversation; not adopted. Incoai; 4-bit g64 with DFlash2 draft and vision.

Size
17.4 GB
Status
On disk
Scored runs
14
Smallest Mac
32 GB estimate, 8K context
Results14 scored · 95% intervals
SuiteScore95% interval PassedRuntime · profileRecorded
epistemicsqwen3.8-27b-splash100.0%92.6–100.0Passed 48/48splashstudy-nothink2026-09-28
epistemicsqwen3.8-27b-splash-nothink100.0%92.6–100.0Passed 48/48splashstudy-nothink2026-09-28
groundingqwen3.8-27b-splash84.8%77.5–90.0Passed 106/125splashstudy-nothink2026-09-28
groundingqwen3.8-27b-splash-nothink83.2%75.7–88.7Passed 104/125splashstudy-nothink2026-09-28
nullschemaqwen3.8-27b-splash75.0%63.9–83.6Passed 54/72splashstudy-nothink2026-09-28
nullschemaqwen3.8-27b-splash-nothink51.4%40.1–62.6Passed 37/72splashstudy-nothink2026-09-28
synthesisqwen3.8-27b-splash88.3%77.8–94.2Passed 53/60splashstudy-nothink2026-09-28
synthesisqwen3.8-27b-splash-nothink83.3%72.0–90.7Passed 50/60splashstudy-nothink2026-09-28
toolcallqwen3.8-27b-splash100.0%94.0–100.0Passed 60/60splashstudy-nothink2026-09-28
toolcallqwen3.8-27b-splash-nothink100.0%94.0–100.0Passed 60/60splashstudy-nothink2026-09-28
toolscaleqwen3.8-27b-splash95.0%83.5–98.6Passed 38/40splashstudy-nothink2026-09-28
toolscaleqwen3.8-27b-splash-nothink100.0%91.2–100.0Passed 40/40splashstudy-nothink2026-09-28
worldviewqwen3.8-27b-splash97.5%87.1–99.6Passed 39/40splashstudy-nothink2026-09-28
worldviewqwen3.8-27b-splash-nothink80.0%65.2–89.5Passed 32/40splashstudy-nothink2026-09-28

Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.

The recordHow it was made · how it was served · what went wrong · what happened to it

Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.

FileHF cache incoai/Qwen3.8-27B-Splash
FormatSplash-packed 4-bit g64, 17,355,931,648 B
Where fromdownloaded 2026-09-28
Statuson disk
Hugging Facehttps://huggingface.co/incoai/Qwen3.8-27B-Splash

Where it came from

  • Repo: incoai/Qwen3.8-27B-Splash, snapshot 9d27070b71f7142c6b6025f03ac011d70a73cb48 (HF cache path). Files, summed by stat -L on 2026-10-23: target/*.bin 66 files, 15,159,640,064 B (embedding, head, 64 layers); draft/ 1,266,040,832 B (DFlash2 draft, five layers plus model.bin); vision/ 930,250,752 B; total 17,355,931,648 B. sha256 not recorded here.
  • Date and tool: 2026-09-28, cache files dated 09:23–09:35Z (10:23–10:35 BST), pulled by the Splash fork's own downloader when the model was first served; a lab note says 'installed and hash-verified'.
  • Runtime used to run it: the community M1 fork ghaslbe/splash-apple7-m1-kernels-fork (branch apple7-m1-kernels, snapshot of paperniuk/splash), built from audited source at builds/splash-m1, binary reported as splash-m1 6d45618. Its README lists this package at 38.1 tok/s and unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_M at 31.6 on the supported-models page.
  • Who asked: Alan, 2026-09-28 08:53Z, after the assistant's fleet-impact analysis recommended a maintenance window for any Splash test.

How it was served

labeldates (UTC)binary and versionctx / -np / KV / -fa / --jinja / --reasoning-formatharness profilewhy this label exists
qwen3.8-27b-splash2026-09-28 11:28–12:57splash-m1 6d45618 (fork of Splash)./splash serve --model incoai/Qwen3.8-27B-Splash --max-memory 26Gstudy-nothink, but thinking was ON: Splash ignores chat_template_kwargs.enable_thinkingthe first full battery on the package; the leak was discovered afterwards from reasoning_chars greater than zero
qwen3.8-27b-splash-nothink2026-09-28 14:35–15:01samesame plus --default-reasoning-effort nonestudy-nothinkthe corrected battery with reasoning verified off

Other servings the same day with no run-log label: a depth-swept SSE speed bench, a four-stream concurrency arm, and /v1/judgments on the 36-case decide/1 set (decide__qwen3.8-27b-splash-judgments.jsonl, 36 rows, not in the run log). Seats held: none. A server was still idle on 2026-10-11.

Decisions

  • Roster implication drawn in the lab's notes: the Splash-hosted dense 3.8 wins the authoring seat (free-form parity with Q4_0, 34 against bonsai2's 7.2 tok/s), while constrained or schema work and fan-out stay on llama.cpp. 'Measured, ready to decide'; no adoption is on record.
  • Dated correction of the earlier thinking-ON battery: both labels are kept, with the first kept because it is informative (nullschema 75.0 with thinking against 51.4 without, above Q4_0's 69.4).
  • Not published as a separate model; its two labels are in the qwen3.8-27b write-up's config list.

What went wrong

  • Mislabelled battery. The run log's profile for qwen3.8-27b-splash reads study-nothink although thinking was on; the corrected re-run exists only under -splash-nothink. Anyone comparing the two labels should read them as thinking-on and thinking-off.
  • Metal incident the same day. Stopping a neighbouring seat (agents-a1) aborted inside ggml's Metal teardown, then guardian crashed 46 s later, and every seat fleetview spawned failed to compile Metal libraries until fleetview was bounced; the Splash server, with a precompiled metallib, kept serving.
  • Standing checks: build stamps in the run log are empty for this runtime; no missing-flag problem beyond the thinking default above.

What happened to it

Downloaded 2026-09-28 → benched 2026-09-28 → idle server left (seen 2026-10-11) → package still in the HF cache (read 2026-10-23). No RESTORE entry needed. Re-downloadable from incoai/Qwen3.8-27B-Splash.