Websites · PROJECT NOTES
WebArena-Verified
A separately maintained WebArena evaluation artifact with revised task definitions, structured response checks and offline scoring from captured network traces.
What can you do with it?
Re-score recorded web-agent outputs with deterministic normalization and structural checks, or collect new runs using supported self-hosted sites.
What should you know before using it?
This revises the original WebArena task family. Its 812 full tasks and 258-task Hard subset in v1.2.3 are not independent tasks to add to WebArena. Offline scoring requires the expected response and trace artifacts; new trajectories still need sites and resets. Pin evaluator/data/image versions; scores need not match the original evaluator.
What is verified here?
Runtime: Python >=3.11 with CLI/package or Docker. The evaluator consumes agent responses and recorded network events. Website containers and optional map/Wikipedia downloads are separate. No container, task replay or state reset was executed.
License note: Evaluator code and the v1.2.3 dataset card declare Apache-2.0. Underlying website software and content retain their own terms. Upstream license ↗
This entry was reviewed against original project material. We have not independently run the software or reproduced its results. AI-development claims are author-reported; check upstream for current versions, setup requirements, and licenses.
- Resource type
- Benchmark suite
- Provenance
- Development provenance not established
- Reviewed
- Platforms
- Web