Apps · PROJECT NOTES
OmniACT
A screenshot-to-action-program dataset pairing desktop and web task instructions with UI images, target annotations and reference PyAutoGUI scripts.
What can you do with it?
Train and evaluate visual models that translate a screen and instruction into a mouse-and-keyboard action program.
What should you know before using it?
Static examples and reference-based action metrics, not a packaged desktop execution environment. Evaluation target boxes must not be used as test-time inputs. The dataset card declares MIT; platform tags identify screenshot sources.
What is verified here?
This entry was reviewed against original project material. We have not independently run the software or reproduced its results. AI-development claims are author-reported; check upstream for current versions, setup requirements, and licenses.
- Resource type
- Dataset + benchmark
- Provenance
- Development provenance not established
- Reviewed
- Platforms
- Web, macOS, Windows, Linux