dlt loads data in, dbt transforms it, and drt activates it back out — reverse ETL from your warehouse to the tools your team works in. Declarative YAML, one drt run.
dlt load → dbt transform → drt activate
pip install drt-core # core (DuckDB included)
drt init && drt run
|
Same DX as dbt. If you know |
CI-native. Exit codes and |
|
LLM-native. A built-in MCP server and Claude Code skills let AI tools author and run your syncs. |
Free and open source. Every connector, the CLI, the MCP server, and the sync engine — Apache 2.0, self-hosted, no lock-in. |
What's always free? All connectors, CLI, MCP server, and sync engine. See OPEN_CORE.md for the open core boundary.
No cloud accounts needed — DuckDB + httpbin.org, three commands.
pip install drt-core
mkdir my-drt-project && cd my-drt-project
drt init --template duckdb_to_restThat scaffolds a runnable syncs/duckdb_to_rest.yml. Seed a tiny
DuckDB table and run:
python -c "
import duckdb
c = duckdb.connect('warehouse.duckdb')
c.execute('''CREATE TABLE IF NOT EXISTS users AS SELECT * FROM (VALUES
(1, 'Alice', 'alice@example.com'),
(2, 'Bob', 'bob@example.com'),
(3, 'Carol', 'carol@example.com')
) t(id, name, email)''')
c.close()
"
drt run --dry-run # preview, no data sent
drt run # POST each row to httpbin.org
drt status # check resultsdrt init --template list # see all available templates
drt init --template postgres_to_slack
drt init --template duckdb_to_hubspotEach template prints next-steps for the env vars / source data it needs. See examples/ for the full collection (Discord, Google Sheets, GitHub Actions, MySQL, ClickHouse, BigQuery, …) and docs/connectors/ for per-connector reference.
For a guided wizard that walks you through profile + project setup:
drt init # interactive — picks a source, configures profile, scaffolds projectBoth flows produce the same project shape (drt_project.yml, syncs/,
.drt/). drt sources --detailed and drt destinations --detailed
print every connector's required env vars and a sample YAML stanza —
useful when hand-authoring beyond the templates.
drt init # initialize project
drt list # list sync definitions
drt sources # list available source connectors
drt destinations # list available destination connectors
drt run # run all syncs
drt run --select <name> # run a specific sync
drt run --all # discover and run all syncs
drt run --select tag:<tag> # run syncs matching a tag
drt run --threads 4 # parallel sync execution
drt run --dry-run # dry run
drt run --verbose # show row-level error details
drt run --output json # structured JSON output for CI/scripting
drt run --log-format json # structured JSON logging to stderr
drt run --profile prd # override profile (or DRT_PROFILE env var)
drt run --cursor-value '…' # override watermark cursor for backfill
drt test # run post-sync validation tests
drt test --select <name> # test a specific sync
drt validate # validate sync YAML configs
drt status # show recent sync status
drt status --output json # JSON output for status
drt profile list # list credential profiles in ~/.drt/profiles.yml
drt profile show <name> # show a profile (secrets masked)
drt profile test <name> # verify a profile's source connectivity
drt profile add <name> # interactively add a profile
drt profile remove <name> # remove a profile
drt serve # start HTTP webhook endpoint
drt docs generate --format mermaid # print project DAG as Mermaid
drt deploy github-actions # scaffold a scheduled sync workflow (drt-action + secrets wired)
drt mcp run # start MCP server (requires drt-core[mcp])
drt --install-completion # install shell completion (bash/zsh/fish)
drt --show-completion # show completion scriptGenerate a Mermaid DAG from your local drt_project.yml and syncs/*.yml files:
drt docs generate --format mermaid > dag.mdgraph LR
subgraph Sources
src_bigquery_prod["bigquery_prod<br/><i>bigquery</i>"]
end
subgraph Syncs
sync_users_to_hubspot{{"users_to_hubspot<br/><i>upsert</i>"}}
sync_accounts_to_hubspot{{"accounts_to_hubspot<br/><i>upsert</i>"}}
end
subgraph Destinations
dst_hubspot_contacts["hubspot (contacts)<br/><i>hubspot</i>"]
end
src_bigquery_prod -->|extract| sync_users_to_hubspot
src_bigquery_prod -->|extract| sync_accounts_to_hubspot
sync_users_to_hubspot -->|load| dst_hubspot_contacts
sync_accounts_to_hubspot -->|load| dst_hubspot_contacts
sync_users_to_hubspot -.lookup.-> sync_accounts_to_hubspot
Shell completion is supported for bash, zsh, and fish:
# Recommended: auto-install for your current shell (idempotent)
drt --install-completion
# Or manually add to your shell config (run once from the target shell)
drt --show-completion >> ~/.bashrc # bash
drt --show-completion >> ~/.zshrc # zsh
drt --show-completion > ~/.config/fish/completions/drt.fish # fishNote:
--show-completionoutputs the script for your current shell. Run it from the shell you want to configure. The manual>>append is not idempotent — run it once only.
After installation, restart your shell and tab-complete commands and options.
Connect drt to Claude, Cursor, or any MCP-compatible client so you can run syncs, check status, and validate configs without leaving your AI environment.
pip install drt-core[mcp]
drt mcp runClaude Desktop (~/Library/Application Support/Claude/claude_desktop_config.json):
{
"mcpServers": {
"drt": {
"command": "drt",
"args": ["mcp", "run"]
}
}
}Available MCP tools:
| Tool | What it does |
|---|---|
drt_list_syncs |
List all sync definitions |
drt_run_sync |
Run a sync (supports dry_run + compute_diff for --diff parity) |
drt_run_test |
Run post-sync validation tests (mirrors drt test) |
drt_get_status |
Get last run result(s) |
drt_get_history |
Get recent sync run history |
drt_validate |
Validate sync YAML configs |
drt_get_schema |
Return JSON Schema for config files |
drt_list_connectors |
List available sources and destinations |
drt_dlq |
Inspect a sync's Dead Letter Queue (depth + failed records) |
drt_retry |
Replay a sync's Dead Letter Queue (mirrors drt retry) |
drt_get_manifest |
Machine-readable sync catalog + lineage (drt docs JSON) |
drt_list_profiles |
List credential profiles (name + type, no secrets) |
drt_test_profile |
Connectivity check for a credential profile |
drt_doctor |
Environment diagnostics (mirrors drt doctor) |
Install the official Claude Code skills to generate YAML, debug failures, and migrate from other tools — all from the chat interface.
/plugin marketplace add drt-hub/drt
/plugin install drt@drt-hubTip: Enable auto-update so you always get the latest skills when drt is updated:
/plugin→ Marketplaces → drt-hub → Enable auto-update
Copy the files from .claude/commands/ into your drt project's .claude/commands/ directory.
| Skill | Trigger | What it does |
|---|---|---|
/drt-create-sync |
"create a sync" | Generates valid sync YAML from your intent |
/drt-debug |
"sync failed" | Diagnoses a specific error and suggests fixes |
/drt-troubleshoot |
"drt isn't working" | Walks a full top-to-bottom diagnostic checklist |
/drt-init |
"set up drt" | Guides through project initialization |
/drt-migrate |
"migrate from Census" | Converts existing configs to drt YAML |
Per-connector reference: docs/connectors/ · Discoverable from the CLI:
drt sources --detailed/drt destinations --detailed
| Connector | Status | Install | Auth |
|---|---|---|---|
| BigQuery | ✅ v0.1 | pip install drt-core[bigquery] |
Application Default / Service Account Keyfile |
| DuckDB | ✅ v0.1 | (core) | File path |
| PostgreSQL | ✅ v0.1 | pip install drt-core[postgres] |
Password (env var) |
| Snowflake | ✅ v0.5 | pip install drt-core[snowflake] |
Password (env var) |
| SQLite | ✅ v0.4.2 | (core) | File path |
| Redshift | ✅ v0.3.4 | pip install drt-core[redshift] |
Password (env var) |
| ClickHouse | ✅ v0.4.3 | pip install drt-core[clickhouse] |
Password (env var) |
| MySQL | ✅ v0.5 | pip install drt-core[mysql] |
Password (env var) |
| Databricks | ✅ v0.6 | pip install drt-core[databricks] |
Access Token (env var) |
| Delta Lake | ✅ v0.8 | pip install drt-core[deltalake] |
Storage options (env var) |
| Iceberg | ✅ v0.8 | pip install drt-core[iceberg] |
Catalog properties (env var) |
| SQL Server | ✅ v0.6 | pip install drt-core[sqlserver] |
Password (env var) |
| REST API | ✅ v0.7 | (core) | Bearer / API Key / Basic / OAuth2 |
34 destinations — warehouses, SaaS APIs, cloud storage, files & webhooks (click to expand)
| Connector | Status | Install | Auth |
|---|---|---|---|
| REST API | ✅ v0.1 | (core) | Bearer / API Key / Basic / OAuth2 |
| Slack Incoming Webhook | ✅ v0.1 | (core) | Webhook URL |
| Discord Webhook | ✅ v0.4.2 | (core) | Webhook URL |
| GitHub Actions | ✅ v0.1 | (core) | Token (env var) |
| HubSpot | ✅ v0.1 | (core) | Token (env var) |
| Zendesk | ✅ v0.7 | (core) | Basic (email + API token) |
| Amplitude | ✅ v0.7 | (core) | Project API key (env var) |
| Klaviyo | ✅ v0.8 | (core) | Private API key (env var) |
| Mixpanel | ✅ v0.8 | (core) | Project token / service account |
| Google Ads | ✅ v0.6 | (core) | OAuth2 Client Credentials |
| Google Sheets | ✅ v0.4 | pip install drt-core[sheets] |
Service Account Keyfile |
| PostgreSQL (upsert) | ✅ v0.4 | pip install drt-core[postgres] |
Password (env var) |
| MySQL (upsert) | ✅ v0.4 | pip install drt-core[mysql] |
Password (env var) |
| ClickHouse | ✅ v0.5 | pip install drt-core[clickhouse] |
Password (env var) |
| Parquet file | ✅ v0.5 | pip install drt-core[parquet] |
File path |
| Amazon S3 | ✅ v0.7.9 | pip install drt-core[s3] |
AWS credential chain / env vars |
| Google Cloud Storage | ✅ v0.7.9 | pip install drt-core[gcs] |
Application Default / Service Account Keyfile |
| Azure Blob Storage | ✅ v0.7.9 | pip install drt-core[azure] |
Connection string env / DefaultAzureCredential |
| Microsoft Teams Webhook | ✅ v0.5 | (core) | Webhook URL |
| CSV / JSON / JSONL file | ✅ v0.5 | (core) | File path |
| Jira | ✅ v0.5 | (core) | Basic (email + API token) |
| Linear | ✅ v0.5 | (core) | API Key (env var) |
| SendGrid | ✅ v0.5 | (core) | API Key (env var) |
| Notion | ✅ v0.6 | (core) | Bearer Token (env var) |
| Airtable | ✅ v0.8 | (core) | Personal Access Token (env var) |
| Twilio SMS | ✅ v0.6 | (core) | Basic (Account SID + Auth Token) |
| Intercom | ✅ v0.6 | (core) | Bearer Token (env var) |
| Email SMTP | ✅ v0.6 | (core) | Username / Password (env var) |
| Salesforce Bulk API 2.0 | ✅ v0.6 | (core) | OAuth2 (username-password) |
| Staged Upload | ✅ v0.6 | (core) | Configurable per provider |
| Elasticsearch / OpenSearch | ✅ v0.7.9 | (core) | API key / HTTP Basic (env var) |
| Snowflake | ✅ v0.7 | pip install drt-core[snowflake] |
Password (env var) |
| Databricks Delta Lake | ✅ v0.7.9 | pip install drt-core[databricks] |
Personal Access Token (env var) |
| BigQuery | ✅ v0.8 | pip install drt-core[bigquery] |
ADC / Service Account keyfile |
| Connector | Status | Install |
|---|---|---|
| Dagster | ✅ v0.4 | pip install dagster-drt |
| Prefect | ✅ v0.6 | (core) |
| Airflow | ✅ v0.6 | (core) |
| dbt manifest reader | ✅ v0.4 | (core) |
Upcoming releases → ROADMAP.md (scope, themes, targets) Issue-level tracking → GitHub Milestones Looking to contribute? → Good First Issues
Shipped: now on v0.7.10 — co-writer-safe mirror · Delta Lake + Iceberg sources · PII masking · Klaviyo + Airtable · VS Code extension. Grouped milestones below; the full per-release changelog lives in CHANGELOG.md and GitHub Releases.
| Milestone | Highlights |
|---|---|
| v0.1–0.3 | Core CLI · BigQuery / DuckDB / Postgres sources · REST / Slack / HubSpot destinations · incremental sync · MCP server + AI Skills for Claude Code |
| v0.4–0.6 | 20+ destinations (Sheets, SQL upserts, Notion, Twilio, Salesforce…) · dagster-drt / Airflow / Prefect · drt test · parallel --threads · JSON Schema validation |
| v0.7 ✅ | Production Ready — graceful shutdown · sync history · atomic table swaps · drt doctor · opt-in telemetry · sync.mode: mirror · DLQ + drt profile |
| v0.7.9–0.7.10 ✅ | Cloud destinations (S3 / GCS / Azure / Databricks / BigQuery / Elasticsearch) · Delta Lake + Iceberg sources · tracked & scoped mirror · PII masking · VS Code extension |
Next: v0.8 Cloud Destinations & Growth → v0.9 Enterprise Foundation → v1.0 Stable Release → v1.x Rust Engine
Community-maintained Dagster integration. Expose drt syncs as Dagster assets with full observability.
pip install dagster-drtfrom dagster import AssetExecutionContext, Definitions
from dagster_drt import drt_assets, DagsterDrtResource
@drt_assets(project_dir="path/to/drt-project")
def my_syncs(context: AssetExecutionContext, drt: DagsterDrtResource):
yield from drt.run(context=context)
defs = Definitions(
assets=[my_syncs],
resources={"drt": DagsterDrtResource(project_dir="path/to/drt-project")},
)See dagster-drt README for full API docs (Translator, Pipes support, DrtConfig dry-run, MaterializeResult).
Run drt syncs straight from CI/CD with the official drt-hub/drt-action — no infrastructure, just a few lines of YAML. Trigger on a schedule, on every push, or right after dbt finishes.
- uses: drt-hub/drt-action@v1
with:
select: '*'
extras: postgres
env:
PG_PASSWORD: ${{ secrets.PG_PASSWORD }}Inputs cover sync selection (select), connector extras, profile, dry-run and threads; outputs expose status, succeeded, failed and duration-seconds (plus a step-summary table). Secrets are passed via env: and resolved by drt's *_env keys. See the action README for the secrets pattern and more examples (run-after-dbt, PR preview).
drt is designed to work alongside, not against, the modern data stack:
drt collects no telemetry by default. Opting in helps us understand which sources / destinations / sync modes are actually used, so we can prioritise.
drt config set telemetry.enabled true # opt in
drt config show-telemetry # preview the exact payload that would be sent
drt config set telemetry.enabled false # opt out
DO_NOT_TRACK=1 drt run # universal kill switch — overrides everythingWhen opted in, drt sends one sync_completed event per sync. The only properties we collect are these 9 fields: drt_version, python_version, os, source_type, destination_type, sync_mode, rows_synced, duration_seconds, status. The wire envelope additionally carries event, distinct_id (a per-machine random UUID at ~/.drt/.anonymous_id), timestamp, and api_key. Sync names, model SQL, destination URLs, credentials, and project paths are never transmitted — the allow-list is enforced at the function-signature level in drt/telemetry.py. By default events go to PostHog Cloud (EU region); override with DRT_TELEMETRY_ENDPOINT and DRT_TELEMETRY_API_KEY for self-hosted PostHog or a custom collector.
Note: drt itself never transmits your IP, but the receiving PostHog backend records the TCP source IP as
$ip. See docs/telemetry.md for details and how to disable / substitute the backend.
For full details see docs/telemetry.md.
We welcome contributions of all sizes — from typo fixes to new connectors. drt has a transparent contributor ladder so your work builds toward greater trust and responsibility over time.
- Get started: CONTRIBUTING.md — setup, workflow, and your first connector tutorial
- Pick something to work on: Good First Issues
- Understand how decisions are made: GOVERNANCE.md
- What's free vs. enterprise: OPEN_CORE.md
- Versioning & breaking changes: VERSIONING.md
Every contribution — code, docs, bug reports, ideas — makes drt better. Thank you 💜
Full credits by contribution type (emoji key)
drt is an independent open-source project and is not affiliated with, endorsed by, or sponsored by dbt Labs, dlt-hub, or any other company.
"dbt" is a registered trademark of dbt Labs, Inc. "dlt" is a project maintained by dlt-hub.
drt is designed to complement these tools as part of the modern data stack, but is a separate project with its own codebase and maintainers.
Apache 2.0 — see LICENSE.