Skip to content

Add Docker support for Modal sandboxes #697

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ColeMurray wants to merge 16 commits into
base: main
Choose a base branch
from
Open

Add Docker support for Modal sandboxes #697

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
0a4d3d0
Add Docker support for Modal sandboxes
ColeMurray Jun 1, 2026
b752ffe
Address Docker support review feedback
ColeMurray Jun 2, 2026
0faa0da
Format Docker service timeout error
ColeMurray Jun 3, 2026
73cd23f
Address follow-up review feedback
ColeMurray Jun 3, 2026
f9b102a
Document repo image build planner
ColeMurray Jun 3, 2026
7eb25f2
Make Docker runtime service opt-in
ColeMurray Jun 3, 2026
f90011b
Expose Docker runtime service config
ColeMurray Jun 3, 2026
2a8c762
Remove image profile from runtime service config
ColeMurray Jun 3, 2026
4485505
Make Docker resource overrides opt-in
ColeMurray Jun 3, 2026
1d9ed1a
Simplify repo image build profile resolution
ColeMurray Jun 3, 2026
8456a47
Align Docker support with provider capabilities
ColeMurray Jun 3, 2026
97c01ca
Model sandbox Docker as provider capability
ColeMurray Jun 3, 2026
5611e5c
refactor: source sandbox provider capabilities from shared
ColeMurray Jun 4, 2026
861697f
refactor: carry runtime intent and shared environment across sandbox …
ColeMurray Jun 4, 2026
5a76c13
refactor: validate snapshot env id and inject Docker deploy policy
ColeMurray Jun 4, 2026
093f766
docs: add ADR 0003 for sandbox runtime capabilities
ColeMurray Jun 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/terraform.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,8 @@ jobs:
TF_VAR_modal_workspace: ${{ secrets.MODAL_WORKSPACE }}
TF_VAR_modal_environment: "${{ secrets.MODAL_ENVIRONMENT || 'main' }}"
TF_VAR_modal_environment_web_suffix: ${{ secrets.MODAL_ENVIRONMENT_WEB_SUFFIX }}
TF_VAR_modal_docker_sandbox_cpu: "${{ secrets.MODAL_DOCKER_SANDBOX_CPU || '4' }}"
TF_VAR_modal_docker_sandbox_memory_mb: "${{ secrets.MODAL_DOCKER_SANDBOX_MEMORY_MB || '8192' }}"
TF_VAR_github_client_id: ${{ secrets.GH_OAUTH_CLIENT_ID }}
TF_VAR_github_client_secret: ${{ secrets.GH_OAUTH_CLIENT_SECRET }}
TF_VAR_github_app_id: ${{ secrets.GH_APP_ID }}
Expand Down Expand Up @@ -323,6 +325,8 @@ jobs:
TF_VAR_modal_workspace: ${{ secrets.MODAL_WORKSPACE }}
TF_VAR_modal_environment: "${{ secrets.MODAL_ENVIRONMENT || 'main' }}"
TF_VAR_modal_environment_web_suffix: ${{ secrets.MODAL_ENVIRONMENT_WEB_SUFFIX }}
TF_VAR_modal_docker_sandbox_cpu: "${{ secrets.MODAL_DOCKER_SANDBOX_CPU || '4' }}"
TF_VAR_modal_docker_sandbox_memory_mb: "${{ secrets.MODAL_DOCKER_SANDBOX_MEMORY_MB || '8192' }}"
TF_VAR_github_client_id: ${{ secrets.GH_OAUTH_CLIENT_ID }}
TF_VAR_github_client_secret: ${{ secrets.GH_OAUTH_CLIENT_SECRET }}
TF_VAR_github_app_id: ${{ secrets.GH_APP_ID }}
Expand Down
3 changes: 3 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,7 @@ Every session runs in an isolated Modal sandbox with a full development environm
and UI verification
- **Code-server:** Optional browser-based VS Code connected to the session workspace
- **Web terminal:** ttyd-powered terminal accessible from the session UI
- **Docker:** Optional Docker Engine and Docker Compose support for Modal-backed sandboxes
- **Port tunneling:** Expose up to 10 dev server ports via encrypted tunnels. URLs are available
in-sandbox at `/workspace/.tunnels.env` before `.openinspect/start.sh` runs
([details](docs/HOW_IT_WORKS.md#tunnel-urls-inside-the-sandbox))
Expand Down Expand Up @@ -249,6 +250,8 @@ docker compose up -d postgres redis
- `setup.sh` failures are non-fatal for fresh sessions, but fatal in image build mode
- `start.sh` runs for every non-build session startup (fresh, repo-image, snapshot-restore)
- `start.sh` failures are strict: if present and it fails, session startup fails
- When Docker is enabled, the Docker daemon starts before either hook runs; expose container ports
through sandbox tunnel ports when you need browser access
- Default timeouts:
- `SETUP_TIMEOUT_SECONDS` (default `300`)
- `START_TIMEOUT_SECONDS` (default `120`)
Expand Down
85 changes: 45 additions & 40 deletions docs/GETTING_STARTED.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -361,6 +361,9 @@ modal_token_secret = "your-modal-token-secret"
modal_workspace = "your-modal-workspace"
modal_environment = "your-modal-environment"
modal_environment_web_suffix = "your-modal-web-suffix" # Lowercase letters, digits, dashes; empty for https://workspace--... endpoints
# Optional Docker-enabled sandbox resource overrides; leave unset to use Modal defaults.
# modal_docker_sandbox_cpu = 1
# modal_docker_sandbox_memory_mb = 2048

# Daytona (only required when sandbox_provider = "daytona")
# daytona_api_url = "https://app.daytona.io/api"
Expand Down Expand Up @@ -662,46 +665,48 @@ Enable automatic deployments when you push to main by adding GitHub Secrets.

Go to your fork's Settings → Secrets and variables → Actions, and add:

| Secret Name | Value |
| ------------------------------ | ------------------------------------------------------------------------------------------- |
| `CLOUDFLARE_API_TOKEN` | Your Cloudflare API token |
| `CLOUDFLARE_ACCOUNT_ID` | Your Cloudflare account ID |
| `CLOUDFLARE_WORKER_SUBDOMAIN` | Your workers.dev subdomain |
| `DEPLOYMENT_NAME` | Your deployment name |
| `R2_ACCESS_KEY_ID` | R2 access key ID |
| `R2_SECRET_ACCESS_KEY` | R2 secret access key |
| `WEB_PLATFORM` | `vercel` or `cloudflare` |
| `VERCEL_API_TOKEN` | Vercel API token _(only if `web_platform = "vercel"`)_ |
| `VERCEL_TEAM_ID` | Vercel team/account ID _(only if `web_platform = "vercel"`)_ |
| `VERCEL_PROJECT_ID` | Vercel project ID _(only if `web_platform = "vercel"`)_ |
| `NEXTAUTH_URL` | Your web app URL |
| `MODAL_TOKEN_ID` | Modal token ID |
| `MODAL_TOKEN_SECRET` | Modal token secret |
| `MODAL_WORKSPACE` | Modal workspace name |
| `MODAL_ENVIRONMENT` | Modal environment name (defaults to `main`) |
| `MODAL_ENVIRONMENT_WEB_SUFFIX` | Modal environment web suffix for endpoint URLs; lowercase letters, digits, dashes, or empty |
| `GH_OAUTH_CLIENT_ID` | GitHub App OAuth client ID |
| `GH_OAUTH_CLIENT_SECRET` | GitHub App OAuth client secret |
| `GH_APP_ID` | GitHub App ID |
| `GH_APP_PRIVATE_KEY` | GitHub App private key (PKCS#8 format) |
| `GH_APP_INSTALLATION_ID` | GitHub App installation ID |
| `ENABLE_SLACK_BOT` | `true` to deploy Slack bot, `false` to skip (default: `true`) |
| `SLACK_BOT_TOKEN` | Slack bot token (required if enabled) |
| `SLACK_SIGNING_SECRET` | Slack signing secret (required if enabled) |
| `ANTHROPIC_API_KEY` | Anthropic API key |
| `TOKEN_ENCRYPTION_KEY` | Generated encryption key (OAuth tokens) |
| `REPO_SECRETS_ENCRYPTION_KEY` | Generated encryption key (repo secrets) |
| `INTERNAL_CALLBACK_SECRET` | Generated callback secret |
| `MODAL_API_SECRET` | Generated Modal API secret |
| `NEXTAUTH_SECRET` | Generated NextAuth secret |
| `ALLOWED_USERS` | Comma-separated GitHub usernames (or empty for all users) |
| `ALLOWED_EMAIL_DOMAINS` | Comma-separated email domains (or empty for all domains) |
| `ENABLE_GITHUB_BOT` | `true` to deploy GitHub bot worker (or empty to skip) |
| `GH_WEBHOOK_SECRET` | GitHub webhook secret (required if GitHub bot enabled) |
| `GH_BOT_USERNAME` | GitHub App bot username, e.g., `my-app[bot]` (required if GitHub bot enabled) |
| `APP_NAME` | Optional display name for whitelabeling (default: `Open-Inspect`) |
| `APP_SHORT_NAME` | Optional short label for sidebar header (default: `Inspect`) |
| `APP_ICON_URL` | Optional URL to a custom logo/favicon (default: built-in icon) |
| Secret Name | Value |
| -------------------------------- | ------------------------------------------------------------------------------------------- |
| `CLOUDFLARE_API_TOKEN` | Your Cloudflare API token |
| `CLOUDFLARE_ACCOUNT_ID` | Your Cloudflare account ID |
| `CLOUDFLARE_WORKER_SUBDOMAIN` | Your workers.dev subdomain |
| `DEPLOYMENT_NAME` | Your deployment name |
| `R2_ACCESS_KEY_ID` | R2 access key ID |
| `R2_SECRET_ACCESS_KEY` | R2 secret access key |
| `WEB_PLATFORM` | `vercel` or `cloudflare` |
| `VERCEL_API_TOKEN` | Vercel API token _(only if `web_platform = "vercel"`)_ |
| `VERCEL_TEAM_ID` | Vercel team/account ID _(only if `web_platform = "vercel"`)_ |
| `VERCEL_PROJECT_ID` | Vercel project ID _(only if `web_platform = "vercel"`)_ |
| `NEXTAUTH_URL` | Your web app URL |
| `MODAL_TOKEN_ID` | Modal token ID |
| `MODAL_TOKEN_SECRET` | Modal token secret |
| `MODAL_WORKSPACE` | Modal workspace name |
| `MODAL_ENVIRONMENT` | Modal environment name (defaults to `main`) |
| `MODAL_ENVIRONMENT_WEB_SUFFIX` | Modal environment web suffix for endpoint URLs; lowercase letters, digits, dashes, or empty |
| `MODAL_DOCKER_SANDBOX_CPU` | Optional CPU cores for Docker-enabled Modal sandboxes; unset uses Modal defaults |
| `MODAL_DOCKER_SANDBOX_MEMORY_MB` | Optional memory in MB for Docker-enabled Modal sandboxes; unset uses Modal defaults |
| `GH_OAUTH_CLIENT_ID` | GitHub App OAuth client ID |
| `GH_OAUTH_CLIENT_SECRET` | GitHub App OAuth client secret |
| `GH_APP_ID` | GitHub App ID |
| `GH_APP_PRIVATE_KEY` | GitHub App private key (PKCS#8 format) |
| `GH_APP_INSTALLATION_ID` | GitHub App installation ID |
| `ENABLE_SLACK_BOT` | `true` to deploy Slack bot, `false` to skip (default: `true`) |
| `SLACK_BOT_TOKEN` | Slack bot token (required if enabled) |
| `SLACK_SIGNING_SECRET` | Slack signing secret (required if enabled) |
| `ANTHROPIC_API_KEY` | Anthropic API key |
| `TOKEN_ENCRYPTION_KEY` | Generated encryption key (OAuth tokens) |
| `REPO_SECRETS_ENCRYPTION_KEY` | Generated encryption key (repo secrets) |
| `INTERNAL_CALLBACK_SECRET` | Generated callback secret |
| `MODAL_API_SECRET` | Generated Modal API secret |
| `NEXTAUTH_SECRET` | Generated NextAuth secret |
| `ALLOWED_USERS` | Comma-separated GitHub usernames (or empty for all users) |
| `ALLOWED_EMAIL_DOMAINS` | Comma-separated email domains (or empty for all domains) |
| `ENABLE_GITHUB_BOT` | `true` to deploy GitHub bot worker (or empty to skip) |
| `GH_WEBHOOK_SECRET` | GitHub webhook secret (required if GitHub bot enabled) |
| `GH_BOT_USERNAME` | GitHub App bot username, e.g., `my-app[bot]` (required if GitHub bot enabled) |
| `APP_NAME` | Optional display name for whitelabeling (default: `Open-Inspect`) |
| `APP_SHORT_NAME` | Optional short label for sidebar header (default: `Inspect`) |
| `APP_ICON_URL` | Optional URL to a custom logo/favicon (default: built-in icon) |

**Bulk upload secrets with `gh` CLI:**

Expand Down
30 changes: 21 additions & 9 deletions docs/HOW_IT_WORKS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,7 @@ development environment.
- Package managers: npm, pnpm, pip, uv
- agent-browser CLI + headless Chrome (for browser automation)
- OpenCode (the coding agent)
- Optional Docker Engine and Docker Compose for Modal-backed sandboxes

Open-Inspect supports two backend patterns:

Expand All @@ -153,6 +154,10 @@ Modal is still the only backend with repo-image builds and live filesystem snaps
uses persistent sandboxes instead: the control plane stops the sandbox on inactivity or stale
heartbeat, then resumes that same sandbox later with the same logical sandbox ID and auth token.

Docker support is Modal-only and controlled by the sandbox `dockerEnabled` setting. Docker-enabled
sandboxes use Modal's default CPU and memory requests unless deployment operators set
`MODAL_DOCKER_SANDBOX_CPU` or `MODAL_DOCKER_SANDBOX_MEMORY_MB`.

### Clients

Clients are how users interact with sessions. The architecture is client-agnostic—any client that
Expand Down Expand Up @@ -192,10 +197,11 @@ When you create a session for a repo without an existing snapshot:
1. **Sandbox created**: Modal spins up a new container from the base image
2. **Git sync**: Clones your repository using brokered SCM credentials from the git credential
helper
3. **Setup script**: Runs `.openinspect/setup.sh` for provisioning (if present)
4. **Start script**: Runs `.openinspect/start.sh` for runtime startup (if present)
5. **Agent start**: OpenCode server starts and connects back to the control plane
6. **Ready**: Sandbox accepts prompts
3. **Docker start**: If enabled, starts `dockerd` before repository hooks run
4. **Setup script**: Runs `.openinspect/setup.sh` for provisioning (if present)
5. **Start script**: Runs `.openinspect/start.sh` for runtime startup (if present)
6. **Agent start**: OpenCode server starts and connects back to the control plane
7. **Ready**: Sandbox accepts prompts

### Restore (From Snapshot)

Expand All @@ -210,8 +216,9 @@ When restoring from a previous snapshot:

1. **Restore snapshot**: Modal restores the filesystem from a saved image
2. **Quick sync**: Pulls latest changes (usually just a few commits)
3. **Start script**: Runs `.openinspect/start.sh` for runtime startup (if present)
4. **Ready**: Sandbox is ready almost instantly
3. **Docker start**: If enabled, clears stale Docker runtime state and starts `dockerd`
4. **Start script**: Runs `.openinspect/start.sh` for runtime startup (if present)
5. **Ready**: Sandbox is ready almost instantly

Snapshots include installed dependencies, built artifacts, and workspace state. This is why
follow-up prompts in an existing session are much faster than the first prompt.
Expand All @@ -221,12 +228,17 @@ follow-up prompts in an existing session are much faster than the first prompt.
When starting from a pre-built repo image:

1. **Incremental git sync**: Fast fetch + hard reset to latest branch head
2. **Setup skipped**: `.openinspect/setup.sh` already ran when the image was built
3. **Start script runs**: `.openinspect/start.sh` executes for per-session runtime startup
4. **Ready**: Agent starts once runtime hook succeeds
2. **Docker start**: If enabled, starts Docker before runtime hooks
3. **Setup skipped**: `.openinspect/setup.sh` already ran when the image was built
4. **Start script runs**: `.openinspect/start.sh` executes for per-session runtime startup
5. **Ready**: Agent starts once runtime hook succeeds

If `start.sh` exists and fails, startup fails fast instead of continuing with a broken runtime.

When Docker is enabled, Open-Inspect uses `/opt/docker-data` as Docker's data root. On every boot it
removes stale daemon runtime directories and sockets while preserving images, layers, volumes, and
BuildKit cache so snapshots and repo images can still accelerate Docker-heavy projects.

### When Snapshots Are Taken

- **After successful prompt completion**: Preserves the workspace state
Expand Down
Loading
Loading