- Parameters
- 24B
- Quant
- Q4_K_M
- Size
- 19.0 GB
- Family
- devstral
RecordStatus · runtime · seat
devstral-small-2-24b-b10618
Measured, not seateddenseresidentllama.cpp
Model card ↗Outcome as of 20 September 2026, from the seat registry. The write-up below was made when the model was tested.
Why it was tested
A coding model, tested to see whether coding tuning transfers to structured extraction.
What came of it
The strongest coder on the extraction suites: 100% screening, 79.2% grounding, 62.5% nullschema. Worth noting that none of those suites measure code, which is the category error that made every coding model look mediocre.
Recorded runs3 scored · 95% intervals
| Suite | Score | 95% interval | Passed | Runtime · profile | Recorded |
|---|---|---|---|---|---|
| grounding | 79.2% | 71.3–85.4 | Passed 99/125 | llama.cppextraction | 2026-08-25 |
| nullschema | 62.5% | 51.0–72.8 | Passed 45/72 | llama.cppextraction | 2026-08-25 |
| screening | 100.0% | 94.0–100.0 | Passed 60/60 | llama.cppextraction | 2026-08-25 |
Runs marked unenforced were measured through mlx_lm on a suite that requires constrained decoding. mlx_lm accepts the schema and ignores it, so those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.
Every run log entry for this model (3)
| Finished | Suite | Runtime · profile | Score | Coverage | Grader · dataset · config |
|---|---|---|---|---|---|
| nullschemanullschema/1 | llama.cppextraction | 62.5% | Coverage 72/72 | g d7448e0f5fc3f9afd 3b91280bb40372efc 836a74186efb247a | |
| screeningscreening/1 | llama.cppextraction | 100.0% | Coverage 60/60 | g 0661b49a10be2d2ad 64c97ca2269c29afc d8a2ef703d82163d | |
| groundinggrounding/1 | llama.cppextraction | 79.2% | Coverage 125/125 | g 30b933dcc6a7e2d4d dcb396faf8b83b0ac ed588aa2abdb41ef |