Apps · PROJECT NOTES
SWE-bench
A source-code repair benchmark that asks an agent to resolve a real repository issue, then evaluates its patch with the project's tests.
What can you do with it?
Evaluate coding agents against executable issue-resolution tests and reuse the supported training splits.
What should you know before using it?
Measures repository-level code repair, not GUI operation or complete app generation. Docker evaluation needs substantial resources; identify the dataset variant and harness version in comparisons.
What is verified here?
This entry was reviewed against original project material. We have not independently run the software or reproduced its results. AI-development claims are author-reported; check upstream for current versions, setup requirements, and licenses.
- Resource type
- Benchmark suite
- Provenance
- Development provenance not established
- Reviewed
- Platforms
- Consult upstream documentation