Underdog-Saluki-27B-1.0 · file← Underdog-Saluki-27B-1.0

IQ2-mix

The one we kept: the model itself, squeezed to about 2 bits. Tested against its 4-bit and 1-bit siblings, never seated. ConwayResearch.

Size: 7.9 GB
Status: On disk
Scored runs: 10
Smallest Mac: 16 GB estimate, 8K context

Results10 scored · 95% intervals

Suite	Score	95% interval	Passed	Runtime · profile	Recorded
groundingsaluki-iq2mix-np4-b11552	68.0%	59.4–75.5	Passed 85/125	llama.cppextraction	2026-10-11
needlesaluki-iq2mix-np4-b11552	68.2%	47.3–83.6	Passed 15/22	llama.cppextraction	2026-10-11
nullschemasaluki-iq2mix-np4-b11552	29.2%	19.9–40.5	Passed 21/72	llama.cppextraction	2026-10-11
screeningsaluki-iq2mix-np4-b11552	95.0%	86.3–98.3	Passed 57/60	llama.cppextraction	2026-10-11
synthesissaluki-iq2mix-np4-b11552	83.3%	72.0–90.7	Passed 50/60	llama.cppextraction	2026-10-11
taggersaluki-iq2mix-np4-b11552	100.0%	43.8–100.0	Passed 3/3	llama.cppextraction	2026-10-11
tagger_nullablesaluki-iq2mix-np4-b11552	100.0%	51.0–100.0	Passed 4/4	llama.cppextraction	2026-10-11
toolcallsaluki-iq2mix-np4-b11552	100.0%	94.0–100.0	Passed 60/60	llama.cppextraction	2026-10-11
toolscalesaluki-iq2mix-np4-b11552	100.0%	91.2–100.0	Passed 40/40	llama.cppextraction	2026-10-11
worldviewsaluki-iq2mix-np4-b11552	75.0%	59.8–85.8	Passed 30/40	llama.cppextraction	2026-10-11

Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.

The recordHow it was made · how it was served · what went wrong · what happened to it

Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.

File	`saluki-27b/Underdog-Saluki-27B-1.0-IQ2-mix.gguf`
Format	IQ2-mix GGUF, 7,898,369,152 B (stat)
Where from	downloaded 2026-10-11 from `ConwayResearch/Underdog-Saluki-27B-1.0`, sha256-verified
Status	benched; on disk
Hugging Face	https://huggingface.co/ConwayResearch/Underdog-Saluki-27B-1.0

Where it came from

Repo / file: ConwayResearch/Underdog-Saluki-27B-1.0, Underdog-Saluki-27B-1.0-IQ2-mix.gguf.
Command (file on disk):

REPO="https://huggingface.co/ConwayResearch/Underdog-Saluki-27B-1.0/resolve/main"
"$HOME/eval/tools/pardl.sh" "$REPO/$f" "$DST/$f" 6
shasum -a 256 "$f" # compared to 4a673518b11b1c4445f9b9a9d3c40356f5ba39f6a6260dcb25f3fbfd475d9efb

Revision: resolve/main, unpinned. The LFS oid/sha256 was recorded from the HF tree API.
sha256: 4a673518b11b1c4445f9b9a9d3c40356f5ba39f6a6260dcb25f3fbfd475d9efb, "sha256 OK".
Date: 2026-10-11. File mtime 15:00 local = 14:00Z (ls, 2026-10-23).
Who asked: Alan, "grab saluki then".

How it was served

label	dates (UTC)	binary and version	ctx / -np / KV / -fa / --jinja / --reasoning-format	profile	why this label exists
`saluki-iq2mix-np4-b11552`	2026-10-11 14:52Z → 15:26Z (full gauntlet, stopped); 15:27Z → ~16:02Z (subset)	`llama.cpp-b11552/build/bin/llama-server`, /props `b11552 (23b0202a1)`	`-c 65536 -np 4` (16,384 per slot), `-ctk q4_0 -ctv q4_0`, `-fa on`, `--jinja`, no `--reasoning-format`, `-ngl 999 --cache-ram 0`	extraction	build-stamped label for the 4/2/1-bit comparison, run via `swapseat_gauntlet.sh` with `VACATE=0` (seats stay up)

No --mmproj on either start, so the vision projector was never loaded. The 64k / 16k-per-slot window is wider than the card's -c 32768. Never seated. Run-log runtime b11552 (23b0202a1) (runtime_source "props").

Decisions

Downloaded for the compression ladder and the bonsai2 replacement question.
Subset run, a decision made in Alan's absence.
Kept on disk, not seated. "Not a Mac Studio seat... Low priority either way" (SUMMARY.md).
Not published (third-party file).

What went wrong

Stopped gauntlet. The first run (all 17 suites, four in parallel) was killed at 15:26Z for speed. It took several attempts to stop; the final kill loop used STOP/KILL. Cost about 33 minutes. The partial files were left under the same label.
Partial rows published as complete (the tagger, tagger_nullable and needle rows). This should be fixed at the source: void them, or set attempted to the suite size.
Decode under 4-way load is not a speed measurement. The SUMMARY says so, and no clean number exists.
Standing checks: --jinja present; build stamp consistent; partial runs found (3).

What happened to it

Downloaded 2026-10-11 → subset-benched 2026-10-11 → not seated → on disk at models/saluki-27b/Underdog-Saluki-27B-1.0-IQ2-mix.gguf (7,898,369,152 B, checked 2026-10-23). No RESTORE entry needed. Re-downloadable from ConwayResearch/Underdog-Saluki-27B-1.0 (verify the sha256 above).