Suite · sitebuild← Suites
Agentic coding harness
Can a model build a working site in a real project with tools and a build it can run?
- Cases
- 1
- Scored runs
- 1
- Models
- 1
- Top score
- 0.0%
SpecificationMethod · grading · caveat
- Method
- The model gets an Astro project, benchmark data, and tools to list, read, write and build. It works until the build passes and the shapeset is accurate, or until its step budget runs out.
- Grading
- The final state of the filesystem. The build must pass, the shapeset must describe what actually exists, and the pages must render real data.
- Caveat
- A model that does nothing scored 0.91 before deliverable checks were added. Conformance rules pass vacuously when nothing was built.
- Version
sitebuild/1- Cases
- 1
Recorded runs1 runs · sorted by accuracy
Sorted by accuracy. The bar shows the 95 percent confidence interval. Where intervals overlap, the models are not distinguishable on this suite at this number of cases.
Top of the suite Qwen3 Coder 30B 0.0% 95% interval 0.0–79.3
| Model | Score | 95% interval | Passed | Runtime · profile | Recorded |
|---|---|---|---|---|---|
| Qwen3 Coder 30Bqwen3-coder-30b | 0.0% | 0.0–79.3 | Passed 0/1 | llama.cppagentic-fast | 2026-09-07 |
Run log1 logged runs of this suite
Every logged run, newest first (1)
| Finished | Model | Runtime · profile | Score | Coverage | Grader · dataset · config |
|---|---|---|---|---|---|
| qwen3-coder-30b | llama.cppagentic-fast | 0.0% | Coverage 1/1 | g b2c603d6c0d6afb3d 8b54725d2778dc60c 0cac83c8fbc03eec |
Findings1 that name this suite