Software it builds.
Worlds it learns in.

A Windows dictation and meeting-notes application with hotkey voice input, transcription and locally stored note folders.

A container-based benchmark for agents that repair and modify dbt projects, SQL transformations and warehouse outputs.

A generation pipeline and released suite of synthetic, SQL-backed tool environments exposed through MCP for multi-turn agent research.

A multi-domain agent benchmark whose current function-calling edition provides containerized tasks for operating-system commands, databases, knowledge graphs, shopping and text-based household tasks.

A multi-environment evaluation suite that tracks intermediate progress as well as final success across web browsing, tool operation, games and simulated-world tasks.

A tool-use evaluation environment with simulated applications and untrusted tool data, designed to measure task utility and resistance to prompt injection.

A common client/server interface for interactive agent environments, accompanied by evaluation tasks, trajectories and agent-development workflows.

A multi-turn reinforcement-learning stack that connects AgentGym environment servers to rollout generation and policy optimization built on verl.

Cross-platform computer-use demonstrations from the OpenCUA project, accompanied by recording tools and an offline action-matching evaluator.

A personal grocery-price history website with product tables, per-product timelines and charts generated from a maintained CSV.

Aligned text and embodied household worlds where agents can learn task plans in language before attempting corresponding visual interactions.

A benchmark and execution harness for optimizing numerical and algorithmic programs while preserving valid outputs, measured against reference runtimes.

An Android GUI dataset combining element grounding, screen and element functionality descriptions, and instruction-following action chains.

An iPhone and iPad app using Apple's Screen Time controls to restrict selected gambling apps and websites, with check-ins and progress tracking.

The public 1.0 task set for Android software engineering: agents edit Kotlin/Java repositories and have their patches checked by build and test harnesses.

A large offline collection of human Android demonstrations containing natural-language goals, UI screenshots and recorded touch, typing and navigation actions.

An Android-in-the-Wild-derived dataset that adds screen descriptions and action-reasoning annotations to recorded screenshot/action sequences.

Human Android task demonstrations with both overall goals and step instructions, plus screenshots, accessibility trees and structured actions.

A reinforcement-learning framework that exposes Android applications through touchscreen actions, pixel observations and configurable task rewards.

An Android-agent framework with prepared virtual devices, reproducible app tasks, evaluation tooling and the Android Instruct training dataset.

An Android emulator environment with parameterized tasks and state-based rewards across everyday mobile applications.

An Android right-triangle calculator with a diagram, configurable decimal precision and named saved calculations.

A tool-use benchmark with runnable API implementations, evaluation dialogues and separate training data for planning, retrieving and calling tools.

A mobile GUI benchmark built from controllable simulated Android apps, with task-relevant interaction flows, seeded backend data and outcome checks.

A simulated world of connected everyday apps with APIs, persistent task state and executable checks for function-calling and interactive coding agents.

A Stella-based Atari 2600 interface that exposes observations, actions, game scores and episode endings to learning agents.

A live-web research benchmark with realistic information-gathering tasks, public development answers and a SeePlanAct browser-agent implementation.

A browser space-shooter demo from OpenAI's single-prompt GPT-5 coding collection, with the generating prompt and published app files.

An AI-generated browser drum sequencer with a sixteen-step grid, tempo and swing controls, saved patterns and WAV export code.

A 50-task computer-use benchmark for dimensioned 2D drafting and true 3D modeling in AutoCAD, with a public corpus, Windows broker and final-DWG scoring code.

An evaluation harness for language and vision-language agents across existing interactive games, including text adventures and NetHack-based tasks.

A browser platform fighter with shared arenas, online matches and bots that fill empty player slots.

A collection of controlled interactive RL experiments designed to diagnose exploration, memory and other agent capabilities with standardized logging.

A desktop sticky-note application for rich text, file shortcuts, pasted images, code blocks, reminders and tabbed notes.

An interactive text-to-SQL environment with database state, supporting knowledge, a user simulator and executable checks for query and update tasks.

A bioinformatics benchmark with a released notebook execution environment for agents that inspect data, run Python/R/Bash analyses and interpret results.

A 3D graphics-editing benchmark that asks model systems to modify Blender Python code so a starting scene matches a target scene.

A browser-agent safety evaluation toolkit with synthetic websites, behavior datasets and trajectory classifiers for studying whether agents follow harmful requests.

A shared browser environment interface that connects agents to multiple web benchmarks and exposes consistent observations and actions.

An AI-generated developer-platform landing page with a monochrome grid, animated terminal, code tabs, pricing table and signup dialog.

A FreeCAD computer-use benchmark that evaluates screenshot-driven GUI work through executable checks on saved CAD projects and engineering outputs.

A coding-agent application that works in a terminal to inspect repositories, edit files and run development tasks, with documented native Windows support.

An experimental C compiler written in Rust by Claude Opus 4.6, released by Anthropic as an artifact of its agent-team research.

A live-website evaluation framework with everyday task specifications, browser-agent harnesses, trace capture and request-based or trajectory-based scoring.

A native macOS app for debugging MCP servers through tool calls, resource previews and live logs.

A browser-based 3D runner and shooting game in which players change lanes, jump obstacles and shoot targets as levels progress.

A JAX survival-and-crafting world that extends Crafter with roguelike mechanics; Craftax-Classic is a separate simpler mode in the same project.

A procedurally generated survival game for studying exploration, resource collection and crafting, with achievement-based agent evaluation.

A Salesforce-based professional CRM task benchmark with public evaluation code and datasets; its Pro extension adds sales, service, quotation workflows and multi-turn user interactions.

An OpenAI-published generated browser tool that reads a CSV file and configures bar, line, pie or doughnut charts with column mapping, colors and PNG download logic.

An Android media log that ranks films, television, games and books through pairwise comparisons instead of fixed star scores.

Multi-file data-analysis tasks combining payment records, JSON business data and documentation, with a public code-executing agent baseline and answer scorer.

A reconstruction benchmark with public code for agents to explore Plotly+Dash dashboards and reproduce their visual design and interactive behavior; its DashboardMimic data is gated.

A native macOS interface for Apple's container platform, with container controls, image management, logs and resource monitoring.

A customizable first-person 3D game platform for visual navigation, memory and control tasks, built on the Quake III/ioquake3 engine lineage.

A C++/Lua engine for building layered two-dimensional grid worlds with multiple agents, text maps and programmable behavior.

A screenshot-to-HTML benchmark with reference webpages, generated predictions and automated visual metrics for evaluating front-end code generation.

An Ubuntu desktop benchmark for professional creative and engineering workflows, with 386 standard tasks and 152 tasks that add phased simulated-user interaction.

An iPhone app for identifying dogs from photos and saving them, names and notes in a personal collection.

A Doom-inspired browser game controlled by scrolling or swiping, with automatic firing, monsters, weapon upgrades and news-headline plaques.

An execution-tested data-science code-generation benchmark covering 1,000 Python problems across seven widely used libraries.

An alpha Rust motion-graphics and compositing app that combines layers, keyframes and effects, with native project files and documented Lottie/media export.

A professional engineering benchmark with GUI and CLI workflows across CAD, simulation, manufacturing, building design, electronics and 3D content creation.

A single-file browser maze game attributed by OpenAI to its AI-generated coding collection, with keyboard navigation, timing and shortest-path comparison.

An experimental Rust browser engine and desktop shell produced during Cursor's research on long-running, collaborating coding agents.

An alpha Rust video editor with a multitrack timeline, trimming, titles, color and audio controls, plus documented media-export and timeline-interchange paths.

A browser-based 3D wood-splitting toy built around scanned wood, an axe and a stump, with interactive splitting and a rotating view.

A futuristic browser racing game with hovercraft, boost and air-brake controls, time trials and selectable race modes.

A multimodal assistant benchmark whose tasks combine reasoning, web research, file handling and tool use, released as questions with supporting attachments.

An asynchronous benchmark on Meta's Agents Research Environments, where agents use simulated apps while events, user requests and time constraints change the shared state.

A game-world generation benchmark with public prompts and Blender scene artifacts, accompanied by Code2Games code for scene analysis, asset placement and gameplay-world construction.

A benchmark for generating complete playable 2D Godot projects from specifications, evaluated through launch checks, replayed player inputs and multimodal rubric scoring.

An executable benchmark of scoped Godot development tasks involving gameplay logic, interfaces, graphics and animation, with starting projects and engine-level validation.

A configurable grid-world game engine using a YAML description language to define mechanics, observations and single- or multi-agent tasks.

An offline Android dataset of cross-app navigation demonstrations, combining screenshots and actions with instructions and semantic reasoning annotations.

A GUI video dataset and offline benchmark for understanding interface content, changes and operation sequences across desktop, mobile, web and XR recordings.

The released navigation component of GUICourse: screenshot/action data for single-step and multi-step website tasks and Android interactions.

A Video Game Description Language framework for evaluating planners and generating game levels or rules across multiple games.

A common interface and reference environments for single-agent reinforcement learning, continuing the maintenance lineage of OpenAI Gym.

A functional JAX environment library with implemented classic-control, diagnostic and small-game tasks, exposing explicit state, randomness and parameters.

An execution framework for agent evaluations and training rollouts, with reusable task formats and local or remote sandbox backends.

A macOS menu-bar app that displays usage windows and reset times for multiple Claude Code and Codex accounts.

A benchmark of webpage interaction transitions, pairing before-and-after screenshots with action annotations and public generation and evaluation code.

An interactive coding environment framework that returns execution feedback during Bash, SQL, Python and capture-the-flag tasks.

An iPhone Simulator benchmark with 26 purpose-built apps, a shared fictional user profile and tasks that span apps and personal context.

A video-to-webpage reconstruction benchmark with user-interaction recordings, static assets, action sequences and public code that evaluates generated pages.

A Python interface connecting agents to interactive-fiction games, with access to game state and controls for reproducible play.

A JAX suite of interactive logic games, packing, routing and other decision-making environments with configurable problem instances.

A browser-agent benchmark for researching sources and producing Google Docs, Sheets and Slides artifacts through multi-step workflows.

A Windows vocabulary tool that displays word-and-definition cards as a floating overlay while other applications remain in use.

A browser-accessible reconstruction of a 1990s multiplayer text adventure, running preserved world scripts on a newly implemented engine.

An alpha Rust photo library and RAW developer with catalog organization, nondestructive adjustments, masking and image-export workflows.

An Android task-execution and evaluation testbed that checks whether agent traces reach annotated essential application states.

A macOS computer-use benchmark combining ported desktop tasks with macOS-specific workflows in an Apple Silicon virtual-machine environment.

A grid-world engine and reference scenarios for many interacting agents, including battle and pursuit settings exposed through PettingZoo APIs.

An OpenAI-published generated browser tool that turns typed Markdown into a slide preview, using lines of three hyphens as slide breaks.

A multi-agent social-evaluation suite pairing interactive worlds with background agent populations to test generalization to unfamiliar partners.

An offline dataset and benchmark of human web-task demonstrations, pairing instructions and action sequences with captured webpage information.

An agentic-search benchmark for long-horizon web research and citation-backed synthesis, with a released judge-agent framework and versioned task-specific evaluators.

A Minecraft research suite that combines survival, resource-gathering, creative and playthrough tasks with a separate knowledge collection.

A Python interface to Minecraft research tasks, with versioned environments and a separate collection of human demonstrations.

A configurable collection of small grid worlds for navigation, object manipulation and instruction following, including the BabyAI environments.

A NetHack-based sandbox for building compact game tasks, from navigation and object use to procedurally generated challenges.

A collection of compact browser tasks for clicking, typing, selecting and manipulating interface elements, exposed through Gymnasium and JavaScript interfaces.

A minimal 3D indoor world simulator for navigation and goal-directed learning, with configurable rooms, objects and visual observations.

A Gym-style research environment where agents modify machine-learning methods, run experiments and receive task-specific feedback.

A mobile-agent benchmark library with task-specific setup, completion checks and resets for everyday Android apps such as calendar, contacts and file management.

A multi-objective multi-agent environment suite whose PettingZoo-style interfaces return reward vectors, including board games, cooperative physics tasks and resource-allocation simulations.

Particle-world environments for communication, cooperation and competition, continuing the MPE environments formerly distributed with PettingZoo.

An Android Japanese keyboard with split layouts, kana-kanji conversion, developer shortcuts and optional voice-input processing.

An iPhone/iPad habit tracker centered on three habits, daily check-ins, reminders and progress history.

A browser hover-car exploration game set in a dense neon city, with street-level driving, flight and time trials.

A Gymnasium interface to NetHack with procedurally generated dungeons, inventory management and long sequences of consequential actions.

A desktop Markdown reader and editor with linked notes, separate annotation files, local vaults and configurable agent integrations.

An office-workflow benchmark in which language agents coordinate document, spreadsheet, email and calendar operations through structured tool commands.

A screenshot-to-action-program dataset pairing desktop and web task instructions with UI images, target annotations and reference PyAutoGUI scripts.

An Android home-screen widget for recording activity timestamps, counting presses and tracking on/off intervals, with history and CSV export.

A generated browser drawing app with pen and eraser tools, color choices and PNG image export.

A live-web benchmark with tasks across public websites, trajectory submissions and WebJudge evaluation, maintained separately from the offline Mind2Web dataset.

A browser-hosted ecosystem of configurable simulated apps for training UI agents and measuring reliability across interface and content variations.

A framework for packaging interactive environments behind a shared step/reset API, with Docker deployment, reward interfaces and training integrations.

A framework of game environments and algorithms for single-agent and multi-agent learning, search and planning with different information structures.

A cross-platform GUI grounding pretraining corpus combining newly captured desktop/web interfaces with reused mobile and web datasets.

An Android safety-research framework combining state/rule checks and a vision-language judge, with mobile task scenarios and recorded-risk trajectory evaluation.

A desktop-agent environment with application workflows, file operations and executable task checks; the original paper introduced 369 tasks.

A distinct 108-task computer-use benchmark for long-horizon workflows spanning Ubuntu desktop applications and web services.

Human reference trajectories and efficiency scoring for OSWorld tasks, measuring how many agent actions a successful workflow requires.

An alpha Rust PDF workbench for viewing, searching, annotations, forms and page organization, with native applications and command-line tools.

A multi-agent environment library with turn-by-turn Agent Environment Cycle and parallel-action APIs, covering cooperative and competitive games.

An alpha Rust image editor with layered documents, masks, brushes, editable type, adjustments and PSD/PSB import and saving.

A native macOS companion for browsing Claude Code sessions, inspecting tool activity and reviewing conversation history.

A native iPhone and iPad app that calculates coffee-to-water amounts and presents a step-by-step pour-over brewing guide.

A suite of sixteen procedurally generated visual games designed to separate learning on familiar levels from generalization to unseen levels.

A community rewrite of Procgen with C++ game implementations and a CEnv interface for using compiled worlds from Python.

An RL execution framework with an Ocean collection of native game and simulation implementations, alongside environment integration and training infrastructure.

A browser real-time strategy demo combining base construction, unit control, abilities and collectible army upgrades.

An iPhone personal-review log with custom fields, photos, pairwise rankings, search and JSON import/export.

A Windows command-line utility for extracting KartRider .rho and .rho5 asset archives, with a published standalone executable.

A card-game environment toolkit exposing observations, legal actions and trajectories for reinforcement learning and search under imperfect information.

An iPhone teleprompter with script editing, text/Markdown import, adjustable scrolling, mirror modes and Bluetooth remote controls.

An executable benchmark for generating standalone Python programs that perform data-driven scientific analyses, with task assets and direct or containerized evaluation.

A scientific desktop environment and benchmark covering molecular visualization, astronomy, GIS, algebra, theorem proving and scientific document workflows.

A text-based simulated world where agents manipulate objects and perform elementary science tasks such as heating, classification and electrical experiments.

A Python command-line utility that examines printing-film PDFs and exports annotated TIFF images and a text report.

A screenshot-based GUI grounding benchmark that asks a model to locate the control named by an instruction across mobile, desktop and web interfaces.

A GUI grounding benchmark using high-resolution screenshots of professional software, including development tools, creative applications, CAD and office suites.

A native iPhone and iPad ledger for recording gifts and cash exchanged at celebrations, organized by person and occasion.

An Android terminal and development workspace combining an on-device shell with editor, browser and AI interaction panes.

An AI-generated product-launch website for a fictional smartwatch, with a specification section, product options and a simulated preorder form.

A browser image tool for composing social-media cards with selectable aspect ratios, background colors, crop previews and downloadable image formats.

An Android-agent evaluation framework with single-app and cross-app tasks in English and Chinese, trajectory annotations and an automated scoring pipeline.

A single-player browser arcade game in which a ship moves horizontally and shoots descending aliens.

An executable desktop benchmark for data-science and engineering workflows that combine code, GUI actions and enterprise tools such as dbt, Airbyte and BigQuery.

A real-world spreadsheet-editing dataset and benchmark with multiple workbook test cases for each user instruction.

A distinct benchmark for end-to-end business spreadsheet generation, debugging and visualization across complex multi-sheet workbooks.

A maintained Gym Retro fork connecting classic-game emulators to Gymnasium, with integrations defining rewards, state variables and episode endings.

A ToolBench-derived environment that combines cached API responses with model-based simulation and revised tool-use evaluation procedures.

A portable Windows sticky-note application with checkboxes, reminders and notes that can follow a selected application window.

A source-code repair benchmark that asks an agent to resolve a real repository issue, then evaluates its patch with the project's tests.

Scale AI's long-horizon repository-repair benchmark, with released public task containers, issue specifications and patch-verification tooling.

Executable Python-repository tasks and test verification for training software-engineering agents and solution verifiers.

A software-engineering benchmark whose public Diamond release combines application-code fixes with choosing among implementation proposals from real freelance work.

A toolkit that builds repository execution environments and synthesizes code-repair tasks by introducing changes that fail existing tests.

A versioned collection of executable terminal tasks with containerized starting environments, reference solutions and automated checks for agent outcomes.

A collection of competitive and cooperative text-based games with a shared string-observation/string-action interface for training and evaluating language-model agents.

A toolkit for generating and playing text adventures with controlled task difficulty, state tracking and rewards for learning agents.

A self-hosted simulated software company where agents use internal web services, files, code and conversations with simulated coworkers to complete workplace tasks.

A Three.js browser port of Quake, with the original game's shareware episode and a public source repository.

A Roman-themed browser tic-tac-toe artifact from OpenAI’s generated examples, with local two-player and computer-opponent modes.

A web-agent benchmark with controlled Wiki, News and Shop environments rendered in multiple historical interface styles.

A generated browser app for moving task cards between Todo, Doing and Done, with local persistence, search and work-in-progress limits.

A stateful tool-use evaluation environment with simulated conversations, dependent tool actions and checks on intermediate and final world states.

An iPhone and iPad utility for Milwaukee freeway travel times, incidents, favorite routes and commute alerts.

An AI-generated browser typing game where falling words, rising difficulty, a score and an accuracy counter form the interaction.

An offline desktop GUI benchmark with publicly released element-grounding and layout-grounding screenshots, annotations and evaluation code.

A Unity SDK and Python interface that turn game and simulation scenes into interactive environments for reinforcement learning, imitation learning and multi-agent research.

A Pygame-generation dataset and evaluation toolkit with natural-language requirements, reference code and model-based scoring of generated code, screenshots and short gameplay recordings.

An alpha Rust vector editor with paths, shapes, text, layers and native/SVG/PDF document workflows, distributed as desktop and WebAssembly builds.

A terminal-session manager with a native Mac app and browser interface for viewing output and interacting with running shells remotely.

An iPhone and iPad app that automatically re-encodes videos into smaller files and supports background processing and sharing.

Instructional-video GUI benchmark data for creative software, separating high-level planning, subtask planning and individual mouse/keyboard actions.

A household activity simulator that executes action programs, with a Unity visual environment and a separate evolving scene-graph backend.

A web-agent benchmark whose shopping, classifieds and forum tasks require visual information alongside text.

A static multimodal web benchmark covering page understanding, OCR, element localization and action grounding from website screenshots.

A configurable first-person game environment exposing Doom-engine observations and controls for visual reinforcement learning and agent evaluation.

Self-hostable websites and task evaluators for agents working across shopping, forums, content management and software collaboration.

A separately maintained WebArena evaluation artifact with revised task definitions, structured response checks and offline scoring from captured network traces.

A public collection of live-website task instructions spanning reading, account changes and file operations, accompanied by published evaluation results.

A distinct task suite built on WebArena's websites, emphasizing tedious workflows that require retaining many details, performing calculations and tracking information across pages.

A web-code dataset pairing rendered page screenshots with HTML/CSS and layout information, accompanied by a screenshot-to-code evaluation pipeline.

A benchmark for generating interactive, multi-file websites from instructions, with browser-agent functional checks and appearance evaluation.

A dataset and benchmark for conversational web navigation, with multi-turn human demonstrations, page observations and action-prediction evaluation tools.

A self-hostable shopping benchmark with four different stores, designed for cross-shop product comparison, vague requirements, compatible alternatives and simulated checkout.

A simulated shopping website where agents search, inspect and select products to satisfy natural-language requests, with HTML and simplified text interfaces.

A public diagnostic dataset for webpage perception, HTML code tasks and image–HTML understanding, with eight released evaluation subsets.

A browser-agent implementation and benchmark that combine screenshots and textual information to carry out tasks on real websites using Selenium.

A Windows desktop-agent benchmark and environment for application, browser and operating-system tasks, with local and Azure execution paths.

A ServiceNow-based task suite for evaluating agents on workplace workflows such as searching knowledge bases, filling forms and filtering records.

A workplace-agent benchmark with sandbox databases and read/write tools for tasks such as email, scheduling and project management, evaluated by the resulting state.

Desktop and web workflow benchmark data with varied starting states, plus a public Windows agent implementation for planning and self-correction.

A JAX grid-world suite for meta-reinforcement learning where composable rules and goals change the dynamics an agent must discover.

A customer-service simulation framework with domain policies, executable tools and simulated users; the current repository extends τ²-Bench into τ³ text, voice and knowledge tasks.