Suite · tools← Suites
Tool and skill selection
Given a request and a set of tools, does the model pick the right one and chain correctly?
- Cases
- 15
- Scored runs
- 22
- Models
- 22
- Top score
- 100.0%
SpecificationMethod · grading · caveat
- Method
- The model is offered tool definitions and asked to complete a task. Chaining matters because entities have to survive from one call to the next.
- Grading
- A case passes when the correct tool is selected and the arguments carry the right entities.
- Caveat
- Superseded for discrimination by toolscale, which varies difficulty instead of holding it fixed.
- Version
tools/1- Cases
- 15
Recorded runs22 runs · sorted by accuracy
Sorted by accuracy. The bar shows the 95 percent confidence interval. Where intervals overlap, the models are not distinguishable on this suite at this number of cases.
Top of the suite Agents-A1 35B 100.0% 95% interval 79.6–100.0
| Model | Score | 95% interval | Passed | Runtime · profile | Recorded |
|---|---|---|---|---|---|
| Agents-A1 35Bagents-a1-35b | 100.0% | 79.6–100.0 | Passed 15/15 | llama.cppagentic-fast | 2026-08-24 |
| Bonsai-27B (Q1_0)bonsai-27b-1bit | 100.0% | 79.6–100.0overlaps the top interval | Passed 15/15 | llama.cppagentic-fast | 2026-09-14 |
| bonsai-27b-mlx2-cortext | 100.0% | 79.6–100.0overlaps the top interval | Passed 15/15 | mlxagentic-fast | 2026-09-14 |
| bonsai-27b-mlx2-m4pro | 100.0% | 79.6–100.0overlaps the top interval | Passed 15/15 | mlxagentic-fast | 2026-09-14 |
| BTL-4 Compact (IQ2_XXS)btl-4 | 100.0% | 79.6–100.0overlaps the top interval | Passed 15/15 | llama.cppagentic-fast | 2026-09-15 |
| BTL-4 (Q4_K_M)btl-4-q4km | 100.0% | 79.6–100.0overlaps the top interval | Passed 15/15 | llama.cppagentic-fast | 2026-09-15 |
| Gemma 4 E2Bgemma4-e2b | 100.0% | 79.6–100.0overlaps the top interval | Passed 15/15 | llama.cppagentic-fast | 2026-08-24 |
| gemma4-e4b | 100.0% | 79.6–100.0overlaps the top interval | Passed 15/15 | llama.cppagentic-fast | 2026-08-24 |
| k2-horizon-7b | 100.0% | 79.6–100.0overlaps the top interval | Passed 15/15 | llama.cppextraction | 2026-09-09 |
| qwen3-next-80b-reap-ag | 100.0% | 79.6–100.0overlaps the top interval | Passed 15/15 | llama.cppagentic-fast | 2026-09-08 |
| Qwen3.8 27Bqwen3.8-27b | 100.0% | 79.6–100.0overlaps the top interval | Passed 15/15 | llama.cppagentic-fast | 2026-08-24 |
| Qwen3.8-9B-Distill (Q8_0)qwen3.8-9b-distill | 100.0% | 79.6–100.0overlaps the top interval | Passed 15/15 | llama.cppagentic-fast | 2026-09-14 |
| Ternary-Bonsai-27B (prism-ml)ternary-bonsai-27b | 100.0% | 79.6–100.0overlaps the top interval | Passed 15/15 | llama.cppagentic-fast | 2026-09-14 |
| Bonsai-8B (Q1_0)bonsai-8b-1bit | 93.3% | 70.2–98.8overlaps the top interval | Passed 14/15 | llama.cppagentic-fast | 2026-09-14 |
| minicpm5-2b | 93.3% | 70.2–98.8overlaps the top interval | Passed 14/15 | llama.cppagentic-fast | 2026-09-15 |
| qwen3-next-80b-ag | 93.3% | 70.2–98.8overlaps the top interval | Passed 14/15 | llama.cppextraction | 2026-09-07 |
| qwen3.6-35b-ag | 93.3% | 70.2–98.8overlaps the top interval | Passed 14/15 | llama.cppextraction | 2026-09-07 |
| ternary-bonsai-8b | 93.3% | 70.2–98.8overlaps the top interval | Passed 14/15 | llama.cppagentic-fast | 2026-09-17 |
| Gemma 4 12Bgemma4-12b | 86.7% | 62.1–96.3overlaps the top interval | Passed 13/15 | llama.cppagentic-fast | 2026-08-24 |
| K2-Horizon-3.7B (IFM)k2-horizon-3.7b | 86.7% | 62.1–96.3overlaps the top interval | Passed 13/15 | llama.cppextraction | 2026-09-09 |
| qwen-agentworld-35b | 80.0% | 54.8–93.0overlaps the top interval | Passed 12/15 | llama.cppextraction | 2026-09-07 |
| qwen3.8-iq1s | 66.7% | 41.7–84.8overlaps the top interval | Passed 10/15 | llama.cppagentic-fast | — |
Run log22 logged runs of this suite
Every logged run, newest first (22)
| Finished | Model | Runtime · profile | Score | Coverage | Grader · dataset · config |
|---|---|---|---|---|---|
| ternary-bonsai-8b | not recordedagentic-fast | 93.3% | Coverage 15/15 | g 2763675a5c3da564d 33a44812f0ad4c39c d3347a668da25f1d | |
| minicpm5-2b | not recordedagentic-fast | 93.3% | Coverage 15/15 | g 2763675a5c3da564d 33a44812f0ad4c39c 29d5dc1655f1e02d | |
| btl-4-q4km | not recordedagentic-fast | 100.0% | Coverage 15/15 | g 2763675a5c3da564d 33a44812f0ad4c39c a04248bbed30bba3 | |
| btl-4 | not recordedagentic-fast | 100.0% | Coverage 15/15 | g 2763675a5c3da564d 33a44812f0ad4c39c e43140ad3e77d008 | |
| bonsai-27b-1bit | not recordedagentic-fast | 100.0% | Coverage 15/15 | g 2763675a5c3da564d 33a44812f0ad4c39c a9cb56f68c317ce8 | |
| bonsai-8b-1bit | not recordedagentic-fast | 93.3% | Coverage 15/15 | g 2763675a5c3da564d 33a44812f0ad4c39c 79e4f8448168d640 | |
| bonsai-27b-mlx2-cortext | mlxagentic-fast | 100.0% | Coverage 15/15 | g 2763675a5c3da564d 33a44812f0ad4c39c 8c56b2e4e619015f | |
| bonsai-27b-mlx2-m4pro | mlxagentic-fast | 100.0% | Coverage 15/15 | g 2763675a5c3da564d 33a44812f0ad4c39c 98ea18969989ee4e | |
| ternary-bonsai-27b | not recordedagentic-fast | 100.0% | Coverage 15/15 | g 2763675a5c3da564d 33a44812f0ad4c39c 7f71a1d06f005b6d | |
| qwen3.8-9b-distill | not recordedagentic-fast | 100.0% | Coverage 15/15 | g 2763675a5c3da564d 33a44812f0ad4c39c 2411d23420a9fcb7 | |
| k2-horizon-3.7b | not recordedextraction | 86.7% | Coverage 15/15 | g 2763675a5c3da564d 33a44812f0ad4c39c 4b2e40f5222213ec | |
| k2-horizon-7b | not recordedextraction | 100.0% | Coverage 15/15 | g 2763675a5c3da564d 33a44812f0ad4c39c d1b260004a9b283e | |
| qwen3-next-80b-reap-ag | not recordedagentic-fast | 100.0% | Coverage 15/15 | g 2763675a5c3da564d 33a44812f0ad4c39c 3907525a897ced43 | |
| qwen3.6-35b-ag | llama.cppextraction | 93.3% | Coverage 15/15 | g 2763675a5c3da564d 33a44812f0ad4c39c a8b21b992370e7c6 | |
| qwen3-next-80b-ag | llama.cppextraction | 93.3% | Coverage 15/15 | g 2763675a5c3da564d 33a44812f0ad4c39c 91dcd2c830490d0e | |
| qwen-agentworld-35b | llama.cppextraction | 80.0% | Coverage 15/15 | g 2763675a5c3da564d 33a44812f0ad4c39c e56fd9e9b9103b0a | |
| qwen3.8-iq1s | llama.cppagentic-fast | 76.9% | Coverage 86.7% coverage | g d6a32c09e57bea7bd 33a44812f0ad4c39c 02e356bf0012eaf2 | |
| qwen3.8-27b | llama.cppagentic-fast | 100.0% | Coverage 15/15 | g d6a32c09e57bea7bd 33a44812f0ad4c39c 35475e0c17bfc385 | |
| agents-a1-35b | llama.cppagentic-fast | 100.0% | Coverage 15/15 | g d6a32c09e57bea7bd 33a44812f0ad4c39c 555c28b5386478f6 | |
| gemma4-e2b | llama.cppagentic-fast | 100.0% | Coverage 15/15 | g d6a32c09e57bea7bd 33a44812f0ad4c39c 33f8efb324ad39aa | |
| gemma4-12b | llama.cppagentic-fast | 86.7% | Coverage 15/15 | g d6a32c09e57bea7bd 33a44812f0ad4c39c b34ee497223231c8 | |
| gemma4-e4b | llama.cppagentic-fast | 100.0% | Coverage 15/15 | g d6a32c09e57bea7bd 33a44812f0ad4c39c 50b166f799c52d51 |