Qwen3-Next-80B-A3B-Instruct · file← Qwen3-Next-80B-A3B-Instruct
Q6_K
A larger copy, too big to fit in memory, used in August to test running a model from the SSD. Never scored.
- Size
- 65.5 GB
- Status
- Deleted 14 Aug 2026
- Scored runs
- 0
This file never completed a scored run. Its record below says why.
Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.
| File | Qwen3-Next-80B-A3B-Instruct-Q6_K-0000{1,2}-of-00002.gguf, plus a merged single file made here with llama-gguf-split --merge |
| Format | GGUF Q6_K, 2 shards, "65.5 GB" (61.0 GiB) |
| Where from | downloaded ~2026-08-12; repo not on record |
| Status | deleted 2026-08-14 (reconstructed) |
| Hugging Face | not on record |
Where it came from
- Files:
models/qwen3-next-80b-a3b/Qwen3-Next-80B-A3B-Instruct-Q6_K-00001-of-00002.ggufplus-00002-of-00002(path ineval/hypura_test.sh). Total "65.5 GB across 2 shards"; the first shard is 46.3 GB (a lab note, BLOCKER 1). - Repo, revision, sha256, download date: not on record. The shard naming and the sibling IQ4_XS's repo make unsloth plausible. That is a guess from a name and is not recorded as provenance.
- Made here: a merged single file, made because hypura reads only the first shard of a split GGUF and OOMs. The command is recorded generically as
llama-gguf-split --merge model-00001-of-00002.gguf merged.gguf(a lab note). The merged filename, the binary build (b10369 era) and the date are not on record. After merging, hypura read "79.7 B / 61.0 GB".
How it was served
| label | dates (UTC) | binary | ctx / -np / KV | profile | why |
|---|---|---|---|---|---|
| — (no run-log label) | 2026-08-12 | hypura (t8/hypura, hypura/target/release/hypura, commit not on record) over its vendored llama.cpp | hypura run <merged> --prompt …; tier placement automatic | none | streaming test: can a model larger than the Metal working set run from NVMe? (eval/hypura_test.sh) |
The control arm in hypura_test.sh ran vanilla llama-cli -ngl 99 on the same oversized file. Its outcome is not recorded in the runbook. No seat was held.
Decisions
- Downloaded as the streaming subject (reconstructed from the runbook's framing; who asked is not on record).
- Deleted 2026-08-14 with the IQ4_XS, 101 GB in total, when Spumco dropped the model from the fleet (
fleet-models.iniREMOVED block). That 101 GB matches IQ4_XS 39.7 GiB + Q6_K 61.0 GiB: reconstructed, because the block does not list files. - The practical verdict at the time: "IQ4_XS resident at 41.4 t/s vs Q6_K streamed at 10.5 t/s" (a lab note).
What went wrong
- hypura mis-plans sharded GGUF. It read 46.3 GB, planned full GPU residency and OOM'd (
kIOGPUCommandBufferCallbackErrorOutOfMemory). Fixed by merging first. - Its estimator reports "[FAST] — suitable for real-time chat" regardless of the numbers.
What happened to it
Downloaded ~2026-08-12 → merged here → hypura streamed at 10.5 t/s → deleted 2026-08-14 (no RESTORE entry). Later sessions cited the result. Re-downloadable from a Q6_K GGUF repo of this checkpoint; which one is not on record.