Apps · PROJECT NOTES
DevOps-Gym
A containerized software-engineering benchmark covering build configuration, runtime monitoring, issue repair and test generation in Java and Go projects.
What can you do with it?
Evaluate agents on individual DevOps stages or selected composed workflows using task-specific instructions and test scripts.
What should you know before using it?
Pin the task revision: the README reports 704 tasks, the inspected tree has 733 direct task directories, and the frozen Harbor package reports 728; these are not established equivalent runnable counts. Issue-repair and test-generation directories reuse issue IDs. An end_to_end folder name alone does not prove a four-stage pipeline: the inspected gogs cpu-usage instruction asks for monitoring. Published image references were not pulled or executed. README declares MIT, but its linked LICENSE file is absent at this revision.
What is verified here?
Runtime: Docker and the documented Terminal-Bench harness, task-specific images and dependencies, plus model access for agent runs. Composed examples use separate server and workload containers. Repository Python guidance must be reconciled with the selected harness version.
License note: README declares MIT, but its linked root LICENSE file is absent at the inspected revision. Task repositories, bundled data/software and container images retain separate terms; the README label does not establish uniform rights over them.
This entry was reviewed against original project material. We have not independently run the software or reproduced its results. AI-development claims are author-reported; check upstream for current versions, setup requirements, and licenses.
- Resource type
- Environment + benchmark
- Provenance
- Development provenance not established
- Reviewed
- Platforms
- Consult upstream documentation