Apps · PROJECT NOTES
SWE-Lancer
A software-engineering benchmark whose public Diamond release combines application-code fixes with choosing among implementation proposals from real freelance work.
What can you do with it?
Evaluate end-to-end code changes and engineering-manager proposal decisions as separate tracks within one benchmark family.
What should you know before using it?
The public release is a subset of the paper's broader task collection. Diamond's 463 tasks comprise 198 coding and 265 manager tasks; combined and split packages must not be added together. Code is MIT, with underlying application terms separate. Valid original runs disable internet on Linux. Harbor changes prompts/test visibility and omits the monetary metric.
What is verified here?
This entry was reviewed against original project material. We have not independently run the software or reproduced its results. AI-development claims are author-reported; check upstream for current versions, setup requirements, and licenses.
- Resource type
- Benchmark suite
- Provenance
- Development provenance not established
- Reviewed
- Platforms
- Consult upstream documentation