Model family← Models

Flint

Recursive's model for coding agents and tool calling, with 39.8 billion parameters and about 2.5 billion at work on each token. A careful checker and good at leaving a field empty rather than inventing a value, but it seldom used a tool unless told to.

Files tested: 1
Scored runs: 15

OverviewWhy we tested it · what happened · should you use it

How this page was made. The tests run automatically on our Mac Studio, and every score here comes straight from those runs. This overview was written by Claude, an AI model, from the lab's own records: the run log, notes kept at the time, and the working sessions where each decision was made. Quotes are Alan's words, with spelling fixed. How these pages are made.

Why we tested it

Recursive, which publishes on Hugging Face as RecursiveSI, released Flint on 21 October 2026, for coding agents and structured tool calling. It has 39.8 billion parameters but uses about 2.5 billion for each token it writes.

Two days later it looked the strongest new candidate in a while for two jobs: our swap seat, the one slot on our server where models are swapped in to try, and the coding role Qwen3.6-35B-A3B holds, with less of the model at work on each token. Alan: "can we download flint?"

We didn't take its maker's word: 89.7 on SWE-bench Pro v2 looked implausibly high beside its 74.2 on SWE-bench Verified.

What it took

llama.cpp learned Flint's design on release day, too late for the builds we were running, so we built a newer release, b11846, beside our usual one. We used Recursive's own Q4_1 file, which stores the 4-bit weights the model was trained with, so nothing was rounded twice.

A first run, four conversations at once with 16K of context each, was stopped after about five minutes: one of our agents won't load a model with less than 64K of context. Alan set the rule our benchmarks now follow: "we should only have one slot running on any benchmark, once tested we decide context and slots if it's worth it". The run of record gave it one conversation at a time, with 64K.

What happened

A careful checker. Judging whether another model's extractions were right, it scored 98.3% (59 of 60). Answering from the document it was given rather than from what it already knew, 95.8% (46 of 48). Several other models score about as high on these two tests, which no longer separate the best.
Good at saying a value isn't there. Asked for a value the document doesn't hold, it left the field empty rather than invent one 70.8% of the time (51 of 72).
It chose tools well but seldom reached for one unprompted. It picked and chained the right tools in all 15 cases, and chose among many 38 times in 40. On our test of what makes a model reach for a tool it scored 55.0% (33 of 60): 12 of 12 when told a tool was available and 11 of 12 with an if-then rule in its instructions, but 2 of 12 when the tool held the answer and nothing said so, and 1 of 12 when the tool was never mentioned.
Quick to write, slow to read long prompts. It wrote 74.2 tokens a second (median, one conversation, other models loaded). On prompts over 16,000 tokens it read 444.7 a second, so a fresh 32,000-token prompt took about 70 seconds before it answered.
Long conversations cost little extra memory. In the run, with 64K of context allocated, it peaked at 26.4 GB. Started at its full 256K it held 31.4 GB, and still wrote about 77 tokens a second on a short prompt. Only 14 of its layers remember the whole conversation; the other 35 see only the latest 513 tokens.
Run the same way as LFM2.5-24B-A2B, it scored higher on 11 of 15 tests. LFM2.5 led on three tool tests, and they tied on one.

What it taught us

Choosing a tool and deciding to use one are different skills. Flint had the first, and needed telling for the second.
Writing speed isn't the whole cost. Reading long prompts is the cost that bites in agent work.

Should you use it?

For checking extractions or filling structured fields it did well here, if you can run a recent llama.cpp; we used b11846. Check the smallest Mac for the file below.

For agents, write each tool's use as an if-then rule in its instructions, or expect it to answer without the tool. We haven't tested its coding, its maker's headline claim, or run Qwen3.6-35B-A3B the same way, so we can't say which codes better. Nor have we checked llama.cpp's new support for it against Recursive's own results. For loops where calling a tool is always right, LFM2.5-24B-A2B reached for one 60 times in 60, though it also wrote tool calls where none were offered.

The files1 tested · each has its own page

Flint
The only file: Recursive's own Q4_1, holding the 4-bit weights the model was trained with. Tested in full once, one conversation at a time with 64K of context.
Benched 24 Oct 2026 · 26.0 GB · 15 scored runs · smallest Mac 36 GB

The lab's recordEvery decision, with its date · the model's specification

Decisions, in order

2026-10-23 21:17Z — download the official GGUF. Alan: "can we download flint?". The official Q4_1 file was chosen over any community requant because it stores the trained INT4 weights. Downloaded 21:20–21:37Z to models/flint-40b-a2.5b/, pinned to revision 9c47151036bc2c34331142dda04488bceab2bc9f; sha256 verified against the HF LFS oid. See the file's page.
2026-10-23 21:41Z — build llama.cpp b11846 and bench under the new convention. Alan: "yes, follow new guidelines for benchmarking though". New worktree llama.cpp-b11846 at tag b11846 (1a24df163), configured with the same flags as llama.cpp-b11552 so the builds differ only in source. The live binary is not touched.
2026-10-23 — pre-registered (Claude, before any run). Plan:
Benches: the full locked gauntlet (eval/gauntlet.sh, 17 suites, extraction profile, 4 suites in flight) through eval/swapseat_gauntlet.sh on a localhost port, with the swap watchdog, VACATE=0 (beside the full fleet; no seat is stopped). Then decide/1 and edgefill. The gauntlet script orders suites longest-first, so a cheap-first order isn't possible; per-suite results are posted as each lands.
Server: llama.cpp-b11846/build/bin/llama-server -np 4 -c 65536 -ctk q4_0 -ctv q4_0 -fa on -ngl 999 --cache-ram 0 --jinja, 127.0.0.1 only. These are the comparators' flags (saluki SUMMARY.md, 2026-10-11), with --jinja stated explicitly. Peak memory is sampled with footprint -p alongside the script's RSS log.
Comparators: the full np4 gauntlets in the run log, kolibri-1 Q4_0 (b11533) and sakura IQ3_XXS (b11533). Saluki IQ2-mix (b11552) ran a 7-suite subset only. qwen3.6-35b, the role holder, has no full gauntlet: its valid rows are grounding, nullschema and screening (2026-08-27), and its toolcall row (2026-09-09) is the oMLX-served void run. So Flint vs qwen3.6-35b is comparable on 3 suites only, unless qwen3.6-35b is re-run.
Expectations, each falsifiable:
Single-stream decode 60–100 tok/s at short context on the Mac Studio. This is the estimate; fine-grained MoEs reach 22–36% of bandwidth here, per MoE decode reaches a third of dense memory bandwidth, and no runtime fixes it.
Resident memory, measured by footprint, of about 28–32 GB at the gauntlet context.
On the four tool suites (toolcall, tools, toolscale, contract), at or above the best np4 comparator on at least 3. A coding and tool-calling model should beat the generalist comparators there.
Risk: as a reasoning model, it may not honour the extraction profile's thinking-off request. If the harness reports a precondition failure, that suite is a harness result, not a model result, and is recorded as an incident.
What would change the plan: the swap watchdog firing (memory), or the harness preconditions failing on thinking control.
2026-10-23 ~21:47Z — vacate the play seat for Flint (Alan's OK). The box was 13.4 GB into swap with all seats up, and Flint's weights are 26 GB. Asked with three options (vacate / run beside / wait); Alan chose to vacate the seat, the option marked recommended. This replaces the plan's VACATE=0. Ornith (23 GB) is restored by swapseat_gauntlet.sh on any exit.
2026-10-23 22:30–22:35Z — first run started and stopped. flint-q4_1-np4-b11846 started at 22:30Z with the np4 plan. The seat was vacated, RSS was 24.3 GB, and "thinking-off verified" showed on the first 4 suites. It was stopped about 5 minutes in, when Alan pointed out that 4 slots give each request 16K: "hermes won't load a model with less than 64". Ornith was restored at 22:35:37Z ("ornith in seat"). The partial result files under that label (grounding 56, tagger 17, tagger_nullable 20 and needle 5 rows) are not scored.
2026-10-23 22:37Z — plan changed to single slot at 64K (Alan's rule). Alan: "we should only have one slot running on any benchmark, once tested we decide context and slots if it's worth it". Now written into the lab's benchmarking runbook. Replaces the pre-registered server and bench lines above, which are kept as registered:
Server: -np 1 -c 65536 -fa on -ngl 999 --cache-ram 0 --jinja, with KV at llama.cpp's default f16 (no -ctk/-ctv; KV quantisation is a serving choice).
Gauntlet: PAR=1 (one suite at a time). shapegen is excluded ("not fully fleshed out", Alan) and so is sitebuild (dataset KeyError: 'nesting'), leaving 15 suites.
Label: flint-q4_1-np1-c64k-b11846, VACATE=1.
Effect on the expectations:
(1) is now tested directly, because single-slot decode is single-stream.
(2) is measured by RSS: footprint leaves out mmap'd weights (as found with LFM2.5-24B-A2B). With f16 KV at 64K, expect slightly more than the 24.3 GB seen at np4.
(3) now compares a single-slot 64K run against np4 16K comparators. These are different configurations, so it's a weaker test, and it says so when scored. The single-slot LFM2.5 run is the only like-for-like comparator.
2026-10-23 23:46Z — Flint run first, LFM2.5 paused. Alan: "once that's done kick of flint", after the bge-m3 and rerank restarts. The LFM2.5 single-slot run was paused after 4 suites, to resume after Flint under the same label. A one-off script runs the coherence gate as each server comes up (the fleet router runbook: "Any new artefact: coherence gate first").
2026-10-24 02:02Z — gauntlet of record complete; the pre-registered expectations, scored (kept beside the plan above).
Decode 60–100 t/s single-stream: met. Median 74.2 t/s (p10–p90 64.1–75.6), contended (fleet resident).
Resident 28–32 GB: missed low. Peak RSS 26.4 GB at 64K f16 KV. The footprint method named in the plan can't measure mmap'd weights, so RSS was used.
At or above the best np4 comparator on ≥ 3 of the 4 tool suites: missed: 1 of 4.
tools 100.0 vs 86.7 is above.
toolcall 55.0 vs 100, toolscale 95.0 vs 100 and contract 60.4 vs 75.0 are all below.
This compares across configurations (np1 64K vs np4 16K), as stated at 22:37Z. The toolcall gap isn't a configuration effect: it's invocation behaviour by arm (What we learned).
Thinking-off risk: did not occur. "thinking-off verified" on every suite. Coherence was COHERENT.

What we learned, in the lab's words

From the one gauntlet of record, flint-q4_1-np1-c64k-b11846 (single slot, 64K, f16 KV, extraction profile, 15 suites, all complete; coherence COHERENT). Scores are pass rate over cases measured, generated from the run's result files.

Comparators, labelled by configuration:

Like-for-like (single slot, 64K): only LFM2.5-24B-A2B so far (lfm2.5-24b-a2b-qad-q4_0-np1-c64k-b11846, partway through: 4 suites done).
qwen3.6-35b: its 2026-08-27 runs, from the published run log, slots not on record.
Sakura: np4 at 16K, a different measurement, shown for context only.

suite	Flint (np1, 64K)	LFM2.5 (np1, 64K)	qwen3.6-35b	sakura (np4, 16K)
screening	98.3 (59/60)	—	86.7	95.0
epistemics	95.8 (46/48)	—	—	100.0
tagger	82.1 (197/240)	62.1	—	67.1
synthesis	78.3 (47/60)	—	—	56.7
enrichment	77.8 (42/54)	—	—	83.3
grounding	77.6 (97/125)	72.8	82.4	79.2
worldview	75.0 (30/40)	—	—	65.0
nullschema	70.8 (51/72)	—	61.1	33.3
tagger_nullable	60.8 (146/240)	50.0	—	85.0
needle (all 110)	60.0 (66/110)	54.5	—	63.6 (55 measured)
tools	100.0 (15/15)	—	—	86.7
toolscale	95.0 (38/40)	—	—	100.0
contract	60.4 (29/48)	—	—	75.0
toolcall	55.0 (33/60)	—	void	100.0

A strong judge and structured-output model. screening 98.3 is above qwen3.6-35b's 86.7 and near granite-guardian's 100. nullschema 70.8 is above qwen3.6-35b's 61.1 and the best nullschema seen on this rig's single-slot runs so far.
It doesn't consult tools unless told to. toolcall by arm: availability 12/12 and conditional (if-then rule in the system prompt) 11/12, but need 2/12 and unmentioned 1/12.
The tool calls themselves work, so this is invocation behaviour, consistent with an earlier lab finding.
For a tool-calling seat, its prompts would need explicit if-then rules naming the trigger. Sakura and LFM2.5 both consulted tools unprompted (60/60).
Grounding is below the role holder: 77.6 vs qwen3.6-35b's 82.4, but different configurations.
Fast decode, slow long prefill: single-stream decode 74.2 t/s (contended), prefill ~445 t/s on 16–32K prompts. A 32K prompt costs about 70 s cold.
Memory: peak RSS 26.4 GB at 64K. It fits the seat's ~23 GB slot only if Ornith leaves it.
Read (Claude, from one run): a candidate for the judgment/screening role and structured extraction, with tool use behind explicit rules. Not yet shown to beat qwen3.6-35b as a coding driver: no coding suite was run, and qwen3.6-35b has no single-slot 64K gauntlet to compare against.

Why it was tested, in the lab's words

Investigated on 2026-10-23 as the strongest new candidate in a while for the swap seat and the coding-driver role, the job qwen3.6-35b does now, with less active compute. Alan asked for the download: "can we download flint?".

Open questions

Whether llama.cpp's b11846 implementation reproduces the vendor's quality (first-day support).
Whether the vendor's 0.7-point NVFP4-vs-16-bit claim has an equivalent for the Q4_1 GGUF; no 16-bit run is planned on this box.
The incumbents (qwen3.6-35b, ornith) have no single-slot 64K gauntlet, so Flint's real comparison needs them re-run under the same procedure.
Coding ability, the vendor's headline claim, isn't measured by the gauntlet. A coding comparison (e.g. the eval/ HumanEval set) is needed before any coding-driver decision.
decide/1 and edgefill are still to run.

The model in detail

field	value
vendor	Recursive (HF org `RecursiveSI`)
upstream repos	`RecursiveSI/flint` (weights), `RecursiveSI/flint-int4-gguf` (official GGUF)
licence	Apache-2.0
architecture	39.8B total, ~2.5B active per token; 256 routed experts, 6 selected per token, plus a shared expert; 35 sliding-window layers (window 513) + 14 full-attention layers, 4 KV heads; context up to 256K
reasoning	reasoning model; the card says to keep `reasoning_content`
released	2026-10-21
llama.cpp support	`FlintForCausalLM`, PR #30928 (commit `a37cc4610`), first in tag b11846 or earlier; not in b10893 or b11552
vendor-reported scores (unverified)	SWE-bench Verified 74.2, Terminal-Bench 2.1 54.9, SWE-bench Pro v2 89.7