FreshContextA context lab · Fife
Drawing
02 · Benchmarks
Sheet
2 of 3
Rev
A · 2026-10-01

Edge0-35B-A3B preview · file← Edge0-35B-A3B preview

Edge0-35B-A3B preview MLX 4-bit + LoRA + prerouter

The only file: Edge0's 4-bit MLX preview with its prerouter and add-on. Kept, and run as our extraction workers. Vendor download.

Size
19.7 GB
Status
On disk
Scored runs
10
Smallest Mac
36 GB estimate, 8K context
Results10 scored · 95% intervals
SuiteScore95% interval PassedRuntime · profileRecorded
enrichmentedge0-35b77.8%65.1–86.8Passed 42/54edge0extraction2026-09-16
epistemicsedge0-35b97.9%89.1–99.6Passed 47/48edge0study-nothink2026-09-16
groundingedge0-35bunenforced89.6%83.0–93.8Passed 112/125edge0extraction2026-09-16
needleedge0-35b62.7%53.4–71.2Passed 69/110mlx_lmextraction2026-09-18
screeningedge0-35b88.3%77.8–94.2Passed 53/60edge0extraction2026-09-16
screeningedge0-35b-recheck88.3%77.8–94.2Passed 53/60mlx_lmextraction2026-09-17
synthesisedge0-35b81.7%70.1–89.4Passed 49/60edge0extraction2026-09-16
taggeredge0-35b88.3%83.7–91.8Passed 212/240edge0extraction2026-09-16
tagger_nullableedge0-35b91.7%87.5–94.5Passed 220/240edge0extraction2026-09-16
worldviewedge0-35b52.5%37.5–67.1Passed 21/40edge0extraction2026-09-16

Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.

The recordHow it was made · how it was served · what went wrong · what happened to it

Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.

Filemodels/edge0-35b/ (4 safetensors shards + LoRA + prerouter + tokenizer/config, about 20 files incl. two demo media)
FormatMLX 4-bit g64 + adapters, 19,689,863,445 B of safetensors
Where fromdownloaded 2026-09-15 from Edge0/Edge0-35B-A3B-preview
Statuson disk and live: serves the EXTRACT pool, edge0-w1 and edge0-w2, natively
Hugging Facehttps://huggingface.co/Edge0/Edge0-35B-A3B-preview

Where it came from

  • Repo: Edge0/Edge0-35B-A3B-preview (Hugging Face), revision not on record. Page: https://huggingface.co/Edge0/Edge0-35B-A3B-preview.
  • Command (2026-09-15 ~20:55Z, run in the background; killed by a tool timeout, resumed twice):
mkdir -p models/edge0-35b
uv run --with huggingface_hub --with hf_transfer python -c "from huggingface_hub import snapshot_download; \
snapshot_download('Edge0/Edge0-35B-A3B-preview', local_dir='models/edge0-35b', max_workers=6)"

The first launch died with its parent shell at 8.3 GB; restarted detached with huggingface_hub resuming the .incomplete files; the four shards were complete at 21:40Z. The stale .incomplete files (8.3 GB, left in .cache/huggingface/download) are gone from the folder today.

  • Contents and size: four shards model-0000[1-4]-of-00004.safetensors = 19,509,027,517 B; lora_edge0_35b.safetensors 42,413,928 B; prerouter_edge0_35b.safetensors 138,422,000 B; total safetensors 19,689,863,445 B (stat, 2026-10-23; source: stat). Plus tokenizer, configs, README.md, a demo jpg and a 19 MB mp4. File mtimes 15 Sep 21:55-22:39 local (20:55-21:39Z).
  • Identity check: shards 2, 3 and 4 have the same sha256 as mlx-community/Qwen3.6-35B-A3B-4bit (shard 4 a5d0cf03..., shard 2 31dcdb1c..., shard 3 3e66de06...; HF cache blobs on the Mac Studio, compared 2026-10-23). Shard 1 does not (local f7fff28f..., 4,395,019,264 B; the mlx-community shard is 5,288,196,018 B). So the vendor's "base" appears to be the community 4-bit conversion with a modified first shard (reconstructed from the hashes; the vendor's recipe, and what shard 1 contains, are not on record). A byte-for-byte match with a Hugging Face file is evidence of identical content here only because the hash, not just the size, matches.
  • Who / why: Alan, "Worth a run due to new architecture, if the pre-router holds up maybe it does something useful".

How it was served

labeldates (UTC)binary and versionflags that matteredharness profilewhy this label exists
edge0-35b (nine suites: grounding, worldview, epistemics, enrichment, screening, synthesis, tagger, tagger_nullable)2026-09-16 06:45-08:54Zedge0 serve from benchmarks/tools/edge0-venv, edge0 0.1.0 (run log: edge0 0.1.0, runtime_source: evidence)edge0 serve models/edge0-35b. Prerouter and LoRA on by default; no context cap exists; no response_format enforcementextraction for eight suites; study-nothink for epistemics (no --profile was passed, so the suite default applied)the main characterisation after Alan's "How did it score"; one-off script: harness.runner --suite $s --model <server> --model-id edge0-35b --label edge0-35b
edge0-35b (needle)2026-09-18 12:55-14:40Zplain mlx_lm server 0.31.3 (PonyExl3 venv; run log mlx_lm 0.31.3)-m mlx_lm server --model models/edge0-35b; --model-id = the folder pathextractionAlan: "I really want to push edge0 to its limits". Same label as the native rows, different runtime (two measurements under one label)
edge0-35b-recheck (screening)2026-09-17 17:41-17:49Zplain mlx_lm 0.31.3 (PonyExl3 venv)python -m mlx_lm server --model <folder>; harness --model-id <folder> --profile extraction --label edge0-35b-recheckextractiondoes the 2026-09-16 screening number reproduce? (control for the MiniCPM5 baseline anomaly, One baseline was 25 points off and nothing in the stamp could explain it; the other six reproduced exactly)

Runs and probes outside the run log (all on dates UTC):

  • 2026-09-15 20:55-22:11Z, native runtime: prerouter A/B and memory samples (first invocation wrong, "wants the path as the positional").
  • 2026-09-16 06:18-06:28Z: footprint -p curve, plateau 5.3 GB.
  • 2026-09-18 12:07Z, mlx_lm: clean decode probe, 53.5 tok/s, TTFT 0.19 s.
  • 2026-09-18 12:36-12:45Z, mlx_lm: contradiction probe.
  • 2026-09-18 20:42Z, mlx_lm: edgefill probe.
  • 2026-09-18 21:13-21:37Z: mlx_lm vmmap; native edge0 serve from a scratch venv; prerouter on/off TTFT; parallel-worker scaling; prefill curve to 64k.
  • 2026-09-19 00:17Z, native runtime (first fleet bring-up); 01:09Z-2026-09-20 ~07:27Z, mlx_lm seat, 19 GB wired; OOM 07:27Z 2026-09-20.

Seats held (outside the run log):

  • Permanent EXTRACT workers, 2026-09-20 09:39Z to the last record (2026-10-11): edge0 serve models/edge0-35b, once for each worker, from venvs/edge0; under launchd since 2026-09-30 (com.freshcontext.edge0-w1.plist / -w2: KeepAlive, RunAtLoad, ThrottleInterval 30, log fleet/logs/edge0-w{1,2}.log, working directory FreshContext-main/freshengine/FreshLoops). fleetview roster entries edge0-w1 / edge0-w2 (upstream_model: edge0-35b) and pool edge0 (least-busy, failover, POOL_MAX_INPUT_CHARS = 16384 * 4).
  • Footprints recorded: 5.1 GB each in the fleet table of 2026-09-20; 8.4 GB for edge0-w1 in the 2026-10-04 memory notes.

Decisions

  • Downloaded to test a streaming-MoE derivative of the board's best grounder.
  • Kept because it came out best on grounding (112/125) and the cheapest worker per GB; the assignment is reading and extraction, not tool use, judgement or conflicts (family record, decision log).
  • Seated 2026-09-20 as two native workers behind a pool; the mlx_lm seat was tried 2026-09-19 and stood down after an OOM.
  • Guardrails recorded (in the fleet router runbook): curated sources only (distractor 4% at 32k), conflicts resolved upstream, 16K-token input cap enforced at the router.

What went wrong

  1. Mean score quoted as pass rate (2026-09-16). The assistant's inline score tables averaged each row's score field, which is partial credit for some suites. They quoted enrichment 93.2%/94.6%, a tagger partial of 96.4% (n=74) and an assembled "tagger 96.7"; the run log's pass rates are enrichment 77.8% (42/54) and tagger 88.3% (212/240). The 2026-09-16 role assignments ("tagging 96.7", "enrichment 94.6", the fleet table) used the inflated figures; the 2026-09-18 table used the pass rates. The assistant also said "I've quoted three partials today already".
  2. Partial snapshot in the run log: tagger/2 at 83 rows, 100% coverage.
  3. Stale build stamp. Every run-log row for this variant carries llama_build: 0.4.0-dev (build 10893, commit c32d1dabe), a llama.cpp build that served none of them (the runtimes were edge0 0.1.0 and mlx_lm 0.31.3). The runtime / runtime_version fields are right.
  4. Wrong runtime measured, then a wrong correction (2026-09-18). The assistant told Alan his 5.3 GB footprint was wrong (it was 21-31 GB) while measuring mlx_lm, and told him the edge0 rows were "under mlx_lm"; the nine suites were native. It retracted. Cost: about eight hours (13:14Z to 21:16Z) of design discussion on the wrong premise, and Alan's 21:13Z confusion.
  5. Needle run under memory pressure. The needle run started at 21 s per case and ended at about 90 s per case with 17.6 GB of swap; the swap was the mlx_lm seat's 18 GB resident weights plus the 32k KV, together with other seats on the box. Scores were collected and not voided, but the swap cost 4x throughput and the other seats' timing on the box was affected.
  6. Epistemics profile. edge0-35b epistemics ran with study-nothink; the Ornith-9B epistemics run (extraction) and the other families' comparisons use extraction. The 97.9% is not the same profile as 100% on the 9B.
  7. Screening recheck "exact" is aggregate only. edge0-35b and edge0-35b-recheck both passed 53/60, but the pass/fail of two cases differs (scr/correct/5 fail then pass; scr/correct/6 pass then fail) and no response string is byte-identical (0/60), as expected from a different server. It also crossed runtimes (edge0 serve vs plain mlx_lm), so it is not a same-runtime repeat.
  8. The OOM of 2026-09-20 07:27Z (mlx_lm seat): see Decisions in the family record.
  9. --jinja and reasoning format: not applicable (MLX servers). The harness sets chat_template_kwargs itself for the probes (enable_thinking: false).

What happened to it

  • Downloaded 2026-09-15 20:55Z; benched 2026-09-16; probes 2026-09-17/18; mlx_lm seat 2026-09-19/20; native workers from 2026-09-20 09:39Z; launchd from 2026-09-30 06:50Z.
  • On disk and live (EXTRACT seats). Not deleted. The 2026-10-11 safetensors sweep verified it untouched ("edge0-35b... LIVE"), and RESTORE-deleted-20261011-tiersAB.sh records it as not removed. If it were ever removed: re-downloadable from Edge0/Edge0-35B-A3B-preview.