Apps and tools.
Workflows to test.

A Windows dictation and meeting-notes application with hotkey voice input, transcription and locally stored note folders.

A container-based benchmark for agents that repair and modify dbt projects, SQL transformations and warehouse outputs.

A generation pipeline and released suite of synthetic, SQL-backed tool environments exposed through MCP for multi-turn agent research.

A multi-domain agent benchmark whose current function-calling edition provides containerized tasks for operating-system commands, databases, knowledge graphs, shopping and text-based household tasks.

A multi-environment evaluation suite that tracks intermediate progress as well as final success across web browsing, tool operation, games and simulated-world tasks.

A tool-use evaluation environment with simulated applications and untrusted tool data, designed to measure task utility and resistance to prompt injection.

A common client/server interface for interactive agent environments, accompanied by evaluation tasks, trajectories and agent-development workflows.

A multi-turn reinforcement-learning stack that connects AgentGym environment servers to rollout generation and policy optimization built on verl.

Cross-platform computer-use demonstrations from the OpenCUA project, accompanied by recording tools and an offline action-matching evaluator.

A benchmark and execution harness for optimizing numerical and algorithmic programs while preserving valid outputs, measured against reference runtimes.

An Android GUI dataset combining element grounding, screen and element functionality descriptions, and instruction-following action chains.

An iPhone and iPad app using Apple's Screen Time controls to restrict selected gambling apps and websites, with check-ins and progress tracking.

The public 1.0 task set for Android software engineering: agents edit Kotlin/Java repositories and have their patches checked by build and test harnesses.

A large offline collection of human Android demonstrations containing natural-language goals, UI screenshots and recorded touch, typing and navigation actions.

An Android-in-the-Wild-derived dataset that adds screen descriptions and action-reasoning annotations to recorded screenshot/action sequences.

Human Android task demonstrations with both overall goals and step instructions, plus screenshots, accessibility trees and structured actions.

A reinforcement-learning framework that exposes Android applications through touchscreen actions, pixel observations and configurable task rewards.

An Android-agent framework with prepared virtual devices, reproducible app tasks, evaluation tooling and the Android Instruct training dataset.

An Android emulator environment with parameterized tasks and state-based rewards across everyday mobile applications.

An Android right-triangle calculator with a diagram, configurable decimal precision and named saved calculations.

A tool-use benchmark with runnable API implementations, evaluation dialogues and separate training data for planning, retrieving and calling tools.

A mobile GUI benchmark built from controllable simulated Android apps, with task-relevant interaction flows, seeded backend data and outcome checks.

A simulated world of connected everyday apps with APIs, persistent task state and executable checks for function-calling and interactive coding agents.

An AI-generated browser drum sequencer with a sixteen-step grid, tempo and swing controls, saved patterns and WAV export code.

A 50-task computer-use benchmark for dimensioned 2D drafting and true 3D modeling in AutoCAD, with a public corpus, Windows broker and final-DWG scoring code.

A desktop sticky-note application for rich text, file shortcuts, pasted images, code blocks, reminders and tabbed notes.

An interactive text-to-SQL environment with database state, supporting knowledge, a user simulator and executable checks for query and update tasks.

A bioinformatics benchmark with a released notebook execution environment for agents that inspect data, run Python/R/Bash analyses and interpret results.

A 3D graphics-editing benchmark that asks model systems to modify Blender Python code so a starting scene matches a target scene.

A FreeCAD computer-use benchmark that evaluates screenshot-driven GUI work through executable checks on saved CAD projects and engineering outputs.

A coding-agent application that works in a terminal to inspect repositories, edit files and run development tasks, with documented native Windows support.

An experimental C compiler written in Rust by Claude Opus 4.6, released by Anthropic as an artifact of its agent-team research.

A native macOS app for debugging MCP servers through tool calls, resource previews and live logs.

A Salesforce-based professional CRM task benchmark with public evaluation code and datasets; its Pro extension adds sales, service, quotation workflows and multi-turn user interactions.

An Android media log that ranks films, television, games and books through pairwise comparisons instead of fixed star scores.

Multi-file data-analysis tasks combining payment records, JSON business data and documentation, with a public code-executing agent baseline and answer scorer.

A native macOS interface for Apple's container platform, with container controls, image management, logs and resource monitoring.

An Ubuntu desktop benchmark for professional creative and engineering workflows, with 386 standard tasks and 152 tasks that add phased simulated-user interaction.

An iPhone app for identifying dogs from photos and saving them, names and notes in a personal collection.

An execution-tested data-science code-generation benchmark covering 1,000 Python problems across seven widely used libraries.

An alpha Rust motion-graphics and compositing app that combines layers, keyframes and effects, with native project files and documented Lottie/media export.

A professional engineering benchmark with GUI and CLI workflows across CAD, simulation, manufacturing, building design, electronics and 3D content creation.

An experimental Rust browser engine and desktop shell produced during Cursor's research on long-running, collaborating coding agents.

An alpha Rust video editor with a multitrack timeline, trimming, titles, color and audio controls, plus documented media-export and timeline-interchange paths.

A multimodal assistant benchmark whose tasks combine reasoning, web research, file handling and tool use, released as questions with supporting attachments.

An asynchronous benchmark on Meta's Agents Research Environments, where agents use simulated apps while events, user requests and time constraints change the shared state.

An offline Android dataset of cross-app navigation demonstrations, combining screenshots and actions with instructions and semantic reasoning annotations.

A GUI video dataset and offline benchmark for understanding interface content, changes and operation sequences across desktop, mobile, web and XR recordings.

The released navigation component of GUICourse: screenshot/action data for single-step and multi-step website tasks and Android interactions.

An execution framework for agent evaluations and training rollouts, with reusable task formats and local or remote sandbox backends.

A macOS menu-bar app that displays usage windows and reset times for multiple Claude Code and Codex accounts.

An interactive coding environment framework that returns execution feedback during Bash, SQL, Python and capture-the-flag tasks.

An iPhone Simulator benchmark with 26 purpose-built apps, a shared fictional user profile and tasks that span apps and personal context.

A Windows vocabulary tool that displays word-and-definition cards as a floating overlay while other applications remain in use.

An alpha Rust photo library and RAW developer with catalog organization, nondestructive adjustments, masking and image-export workflows.

An Android task-execution and evaluation testbed that checks whether agent traces reach annotated essential application states.

A macOS computer-use benchmark combining ported desktop tasks with macOS-specific workflows in an Apple Silicon virtual-machine environment.

A Gym-style research environment where agents modify machine-learning methods, run experiments and receive task-specific feedback.

A mobile-agent benchmark library with task-specific setup, completion checks and resets for everyday Android apps such as calendar, contacts and file management.

An Android Japanese keyboard with split layouts, kana-kanji conversion, developer shortcuts and optional voice-input processing.

An iPhone/iPad habit tracker centered on three habits, daily check-ins, reminders and progress history.

A desktop Markdown reader and editor with linked notes, separate annotation files, local vaults and configurable agent integrations.

An office-workflow benchmark in which language agents coordinate document, spreadsheet, email and calendar operations through structured tool commands.

A screenshot-to-action-program dataset pairing desktop and web task instructions with UI images, target annotations and reference PyAutoGUI scripts.

An Android home-screen widget for recording activity timestamps, counting presses and tracking on/off intervals, with history and CSV export.

A generated browser drawing app with pen and eraser tools, color choices and PNG image export.

A framework for packaging interactive environments behind a shared step/reset API, with Docker deployment, reward interfaces and training integrations.

A cross-platform GUI grounding pretraining corpus combining newly captured desktop/web interfaces with reused mobile and web datasets.

An Android safety-research framework combining state/rule checks and a vision-language judge, with mobile task scenarios and recorded-risk trajectory evaluation.

A desktop-agent environment with application workflows, file operations and executable task checks; the original paper introduced 369 tasks.

A distinct 108-task computer-use benchmark for long-horizon workflows spanning Ubuntu desktop applications and web services.

Human reference trajectories and efficiency scoring for OSWorld tasks, measuring how many agent actions a successful workflow requires.

An alpha Rust PDF workbench for viewing, searching, annotations, forms and page organization, with native applications and command-line tools.

An alpha Rust image editor with layered documents, masks, brushes, editable type, adjustments and PSD/PSB import and saving.

A native macOS companion for browsing Claude Code sessions, inspecting tool activity and reviewing conversation history.

A native iPhone and iPad app that calculates coffee-to-water amounts and presents a step-by-step pour-over brewing guide.

An iPhone personal-review log with custom fields, photos, pairwise rankings, search and JSON import/export.

A Windows command-line utility for extracting KartRider .rho and .rho5 asset archives, with a published standalone executable.

An iPhone teleprompter with script editing, text/Markdown import, adjustable scrolling, mirror modes and Bluetooth remote controls.

An executable benchmark for generating standalone Python programs that perform data-driven scientific analyses, with task assets and direct or containerized evaluation.

A scientific desktop environment and benchmark covering molecular visualization, astronomy, GIS, algebra, theorem proving and scientific document workflows.

A Python command-line utility that examines printing-film PDFs and exports annotated TIFF images and a text report.

A screenshot-based GUI grounding benchmark that asks a model to locate the control named by an instruction across mobile, desktop and web interfaces.

A GUI grounding benchmark using high-resolution screenshots of professional software, including development tools, creative applications, CAD and office suites.

A native iPhone and iPad ledger for recording gifts and cash exchanged at celebrations, organized by person and occasion.

An Android terminal and development workspace combining an on-device shell with editor, browser and AI interaction panes.

An Android-agent evaluation framework with single-app and cross-app tasks in English and Chinese, trajectory annotations and an automated scoring pipeline.

An executable desktop benchmark for data-science and engineering workflows that combine code, GUI actions and enterprise tools such as dbt, Airbyte and BigQuery.

A real-world spreadsheet-editing dataset and benchmark with multiple workbook test cases for each user instruction.

A distinct benchmark for end-to-end business spreadsheet generation, debugging and visualization across complex multi-sheet workbooks.

A ToolBench-derived environment that combines cached API responses with model-based simulation and revised tool-use evaluation procedures.

A portable Windows sticky-note application with checkboxes, reminders and notes that can follow a selected application window.

A source-code repair benchmark that asks an agent to resolve a real repository issue, then evaluates its patch with the project's tests.

Scale AI's long-horizon repository-repair benchmark, with released public task containers, issue specifications and patch-verification tooling.

Executable Python-repository tasks and test verification for training software-engineering agents and solution verifiers.

A software-engineering benchmark whose public Diamond release combines application-code fixes with choosing among implementation proposals from real freelance work.

A toolkit that builds repository execution environments and synthesizes code-repair tasks by introducing changes that fail existing tests.

A versioned collection of executable terminal tasks with containerized starting environments, reference solutions and automated checks for agent outcomes.

A self-hosted simulated software company where agents use internal web services, files, code and conversations with simulated coworkers to complete workplace tasks.

A generated browser app for moving task cards between Todo, Doing and Done, with local persistence, search and work-in-progress limits.

A stateful tool-use evaluation environment with simulated conversations, dependent tool actions and checks on intermediate and final world states.

An iPhone and iPad utility for Milwaukee freeway travel times, incidents, favorite routes and commute alerts.

An offline desktop GUI benchmark with publicly released element-grounding and layout-grounding screenshots, annotations and evaluation code.

An alpha Rust vector editor with paths, shapes, text, layers and native/SVG/PDF document workflows, distributed as desktop and WebAssembly builds.

A terminal-session manager with a native Mac app and browser interface for viewing output and interacting with running shells remotely.

An iPhone and iPad app that automatically re-encodes videos into smaller files and supports background processing and sharing.

Instructional-video GUI benchmark data for creative software, separating high-level planning, subtask planning and individual mouse/keyboard actions.

A Windows desktop-agent benchmark and environment for application, browser and operating-system tasks, with local and Azure execution paths.

A workplace-agent benchmark with sandbox databases and read/write tools for tasks such as email, scheduling and project management, evaluated by the resulting state.

Desktop and web workflow benchmark data with varied starting states, plus a public Windows agent implementation for planning and self-correction.

A customer-service simulation framework with domain policies, executable tools and simulated users; the current repository extends τ²-Bench into τ³ text, voice and knowledge tasks.