Nemotron-3.5-Lightning-30B-A3B · file← Nemotron-3.5-Lightning-30B-A3B
Q4_K_M
Bartowski's GGUF, and the copy most of the testing ran on. Deleted on 8 September. Bartowski GGUF.
- Size
- 25.5 GB
- Status
- Deleted 8 Sep 2026
- Scored runs
- 3
- Smallest Mac
- 32 GB measured, 8K context
| Suite | Score | 95% interval | Passed | Runtime · profile | Recorded |
|---|---|---|---|---|---|
| groundingnemotron-lightning-30b | 60.0% | 51.2–68.2 | Passed 75/125 | llama.cppextraction | 2026-09-05 |
| nullschemanemotron-lightning-30b | 15.3% | 8.8–25.3 | Passed 11/72 | llama.cppextraction | 2026-09-05 |
| screeningnemotron-lightning-30b | 93.3% | 84.1–97.4 | Passed 56/60 | llama.cppextraction | 2026-09-05 |
Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.
Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.
| File | NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Q4_K_M.gguf, 25,477,403,616 B |
| Format | Q4_K_M GGUF (imatrix, MTP head at Q4_0) |
| Where from | downloaded about 2026-08-13 from bartowski |
| Status | deleted 2026-09-08 |
| Hugging Face | https://huggingface.co/bartowski/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF |
Where it came from
- Repo / file:
bartowski/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Q4_K_M.gguf. Revision: not on record. Size 25,477,403,616 bytes, shown as 25.48 GB and 23.73 GiB bydu. - Command:
models/get-nemotron-lightning.sh. It runs a 6-range parallelpget.sh "$B/$f" "$f" 6fromhttps://huggingface.co/bartowski/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF/resolve/mainintomodels/nemotron-lightning-30b/and writes.dl_complete. Thepget.shhelper lived in a session scratchpad and is not on record. - Why bartowski: the script header says "bartowski's imatrix quants, chosen over ggml-org/unsloth because" they are imatrix-calibrated (ggml-org's are not) and "MTP layers embedded in the quant at Q4_0 -> --spec-type draft-mtp works with no separate model-draft file". Quantised with b10362; the local build was b10369 (script header).
- Date: about 2026-08-13. The router preset comment says "Verified loading and generating 2026-08-13" (
fleet-models.ini), and the harness test and eval battery in the lab notes are dated the same day. The exact time, the sha256 and who asked are not on record.
How it was served
One run-log label. Server details are from the script that launched it and the result manifests, not from the run log.
| label | dates (UTC) | binary | ctx / -np / KV / -fa / --jinja / --reasoning-format | harness profile | why this label exists |
|---|---|---|---|---|---|
nemotron-lightning-30b | 2026-09-05 19:11Z to 19:14Z | llama.cpp/build/bin/llama-server, b10618 (eb25b7263) | --ctx-size 32768 --n-gpu-layers 99 --jinja; -np not set (server default, not on record); KV f16 (default); no -fa (auto); no --reasoning-format; alias nemotron-lightning-30b | extraction (thinking off via chat-template enable_thinking=false, max_tokens 2048, temperature 0, seed 42) | the first harness-v2 look at a model that had held a router preset for weeks and had no run-log row; Alan: "Yeah go take it for a spin" |
Run-log runtime_source for these rows says router; the manifests show endpoint, so this was not the router (see incidents).
Other ways this file was served, none of them in the run log:
| use | when | how | cite |
|---|---|---|---|
| 13 Aug eval battery and Hermes harness tests | 2026-08-13 | spumco-eval-battery.py; Hermes profiles on the router/cortex endpoint | lab notes |
| harness-v1 15-suite board | 2026-08-21 08:06Z | llama.cpp, M1 Max | board/results/2026-08-21T08-06-30-nemotron-lightning-30b.json |
router preset [nemotron-lightning-30b] | about 2026-08-13 to 2026-09-08 | c = 131072, no chat-template kwargs, MTP off, --models-max 1 | fleet-models.ini |
DSH cortex provider model | at least 2026-08-23 to 2026-09-09 | client pinned to the model id | |
| MTP A/B, ctx ladder, llama-bench fleet sweep | dates in family doc | private-port servers | eval/ scripts |
Seat held: none. It was a router preset, never one of the fleet's named seats.
Decisions
- Pulled as the three-way A/B partner of qwen3.6-35b-a3b and agents-a1-35b, and as the LoRA target (see family doc).
- Not made a seat. After the harness failure and then pass, agents-a1-35b stayed the production agent (a lab note).
- Kept through at least 2026-09-07: the 17:34Z triage called its MLX sibling dead weight but left the GGUF ("you have the GGUF and a preset").
- Deleted 2026-09-08 07:48Z on Alan's "Kill Nemotron thinking cap and Gemma dense";
rm -rfof the directory (free space 120 to 195 GB across all four deletions). - Not published as weights (the vendor's own derivative). Its scores are on the site.
What went wrong
- Thinking-on default (2026-09-05). Until 19:10Z every request opened
<think>, so the model reasoned before answering; the preset never setenable_thinking. Found by reading the GGUF chat template; cost was not measured on real traffic, only on a one-line probe. - Wrong sampling and use. The harness ran greedy; NVIDIA's suggestion is 1.0 / 0.95 and the model is aimed at post-training. The assistant itself said the nullschema number "doesn’t mean much" as a capability claim.
- Unproven schema enforcement. The run predates the sentinel probe. The by-arm identity of results (nullable and omittable arms gave the same answers as the forbidden arm) is also what an ignored
response_formatwould look like. The mechanism the assistant described in the conversation (the model writes the string not found into the field instead of leaving it null) was shown from aforbidarm case, where a non-nullable grammar would have forced a string. - Run-log source stamp.
runtime_source: routeris wrong (it was a standalone server); the binary version is right. - First-reading error. "the lowest score anything has ever posted" was contradicted by the 2026-08-24 gemma4-e2b and gemma4-e4b rows.
- Stale references after deletion. The router kept offering the preset until restarted on 2026-09-08, and the DSH provider list named it until 2026-09-09.
- Partial runs: none. Stale build stamps: none found (run-log b10618 matches, "Stay on 10618 for production"). Missing flags:
--jinjawas set; no reasoning format was set (not needed with thinking off).
What happened to it
Downloaded about 2026-08-13 → verified, battery and harness tests 2026-08-13 → v1 board 2026-08-21 → router preset with 131072 context (Aug) → harness-v2 suites 2026-09-05 → deleted 2026-09-08 07:48Z. No RESTORE-deleted-*.sh entry exists. Re-downloadable: run get-nemotron-lightning.sh, which needs the pget.sh helper; or hf download bartowski/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Q4_K_M.gguf. Run-log rows survive the deletion.