Websites · PROJECT NOTES
ClawBench
A live-website evaluation framework with everyday task specifications, browser-agent harnesses, trace capture and request-based or trajectory-based scoring.
What can you do with it?
Evaluate document-informed online workflows and inspect recorded actions, requests and model messages under a chosen corpus, harness and rubric.
What should you know before using it?
Runs touch production websites and may need accounts or external services. Interception blocks only requests matching a task schema; nonmatching requests continue live, and placeholder schemas do not block submissions. A blocked request plus judge approval is not a committed real-world result. Current V1/V2 folders have 152/129 tasks; historical counts and scoring protocols differ.
What is verified here?
Runtime: Package v0.11.0 requires Python >=3.11, Docker/Podman or documented remote-browser setup, and model/judge configuration. Native interception scoring and historical V1 human-reference trajectory judging are separate protocols. Live sites, login and browser restrictions remain external dependencies.
License note: Repository code and inspected official task/V1-trace metadata declare Apache-2.0. This does not license third-party websites or authorize live task actions. Upstream license ↗
This entry was reviewed against original project material. We have not independently run the software or reproduced its results. AI-development claims are author-reported; check upstream for current versions, setup requirements, and licenses.
- Resource type
- Benchmark suite
- Provenance
- Development provenance not established
- Reviewed
- Platforms
- Web