FreshContextA context lab · Fife
Drawing
02 · Benchmarks
Sheet
2 of 3
Rev
A · 2026-10-01

Model← Models

Agents-A1 35B

35B mixture of experts, 3B active, Q4_K_M, 21.0 GB.

Parameters
35B / 3B active
Quant
Q4_K_M
Size
21.0 GB
Family
agents-a1
RecordStatus · runtime · seat

agents-a1-35b

Measured, not seatedMoEresidentllama.cpp

Outcome as of 20 September 2026, from the seat registry. The write-up below was made when the model was tested.

Why it was tested

An agent-tuned MoE, tested on the tool suites where its specialism should show.

What came of it

100% on tools, toolscale and epistemics at 49.4 tok/s. Three perfect scores on three suites that are now saturated, which is a statement about the suites.

Recorded runs4 scored · 95% intervals
SuiteScore95% interval PassedRuntime · profileRecorded
epistemics100.0%92.6–100.0Passed 48/48llama.cppstudy-nothink2026-08-24
grounding71.2%62.7–78.4Passed 89/125llama.cppextraction2026-08-24
tools100.0%79.6–100.0Passed 15/15llama.cppagentic-fast2026-08-24
toolscale100.0%91.2–100.0Passed 40/40llama.cppagentic-fast2026-08-24

Runs marked unenforced were measured through mlx_lm on a suite that requires constrained decoding. mlx_lm accepts the schema and ignores it, so those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.

Every run log entry for this model (4)
FinishedSuiteRuntime · profileScoreCoverageGrader · dataset · config
groundinggrounding/1llama.cppextraction71.2%Coverage 125/125g 30b933dcc6a7e2d4d dcb396faf8b83b0ac 3cb35299a047efb8
toolscaletoolscale/1llama.cppagentic-fast100.0%Coverage 40/40g 1bae1f3e1b606eaed b24abb108c409178c f55610aeacd8ae8c
epistemicsepistemics/4llama.cppstudy-nothink100.0%Coverage 48/48g 2a2f7e4e90d6dd63d 4bce453fd376553cc 2101ffcef80989dc
toolstools/1llama.cppagentic-fast100.0%Coverage 15/15g d6a32c09e57bea7bd 33a44812f0ad4c39c 555c28b5386478f6