Apps · PROJECT NOTES
ADE-bench
A container-based benchmark for agents that repair and modify dbt projects, SQL transformations and warehouse outputs.
What can you do with it?
Evaluate changes to data models and analytics pipelines using task-specific dbt tests and expected-table comparisons.
What should you know before using it?
DuckDB tasks copy a local database into a task container. Snowflake tasks use a real account, create or replace trial databases/users/roles, and leave objects behind; upstream advises a separate test account. Database/project variants differ by task. The frozen Harbor package has 48 DuckDB+dbt tasks and does not represent every task in the current source.
What is verified here?
Runtime: Python >=3.10, Docker/Compose, shared dbt projects and separately distributed DuckDB files. Model credentials depend on the selected agent; the bundled reference-solution agent can check task setup without a model. Source inspection found 60 ready task folders at the pinned revision; readiness is not independent runtime verification.
License note: Repository code is Apache-2.0. The database release and inspected metadata do not establish separate terms for every source dataset; do not assume all warehouse contents share the code license. Upstream license ↗
This entry was reviewed against original project material. We have not independently run the software or reproduced its results. AI-development claims are author-reported; check upstream for current versions, setup requirements, and licenses.
- Resource type
- Environment + benchmark
- Provenance
- Development provenance not established
- Reviewed
- Platforms
- Consult upstream documentation