Skip to content

feat: add safe application env upsert - #4581

Open
agentHits wants to merge 6 commits into
Dokploy:canaryfrom
agentHits:feat/application-env-upsert
Open

feat: add safe application env upsert#4581
agentHits wants to merge 6 commits into
Dokploy:canaryfrom
agentHits:feat/application-env-upsert

Conversation

@agentHits

@agentHits agentHits commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

What is this PR about?

This PR adds a safe partial environment variable upsert API for applications. API and MCP clients can add or update selected application environment variables without reading or replacing the full env block, which avoids deleting existing secrets when the current env is redacted.

The new application.env.upsert mutation reads the current env server-side, merges submitted variables by key, and returns metadata only. It supports dryRun, optional expectedRevision conflict protection, and optional redeploy when the caller also has deployment permission.

This update also exposes direct API routing for the new contract:

  • POST /api/application/env/upsert for the slash route used by MCP and OpenAPI clients.
  • POST /api/application.env.upsert as a compatibility route for the dot-style operation name.
  • OpenAPI metadata now publishes the stable API path as POST /application/env/upsert.

Checklist

Before submitting this PR, please make sure that:

  • You created a dedicated branch based on the canary branch.
  • You have read the suggestions in the CONTRIBUTING.md file https://github.com/Dokploy/dokploy/blob/canary/CONTRIBUTING.md#pull-request
  • You have tested this PR in your local instance. If you have not tested it yet, please do so before submitting. This helps avoid wasting maintainers' time reviewing code that has not been verified by you.

Issues related (if applicable)

Closes #4525

Screenshots (if applicable)

N/A. This is an API/server-side change.

MCP support status

Dependent MCP support is already implemented in Dokploy/mcp#56.

That MCP PR is ready for review, but should remain blocked for merge until this core API lands or maintainers confirm the final API/OpenAPI shape. The intended MCP behavior is server-side partial env upsert only: no full env read, no client-side full env reconstruction, no fallback to application.saveEnvironment, and no raw secret values returned to the client.

I also verified the flow in a self-hosted Dokploy + MCP setup using dryRun, so MCP can send only selected variables without reading or rewriting the full env block.

Verification

Passed locally:

  • git diff --cached --check
  • corepack pnpm --filter=dokploy exec vitest run --config __test__/vitest.config.ts __test__/env/upsert.test.ts __test__/env/application-env-upsert.test.ts
  • CI=true corepack pnpm exec biome check 'apps/dokploy/pages/api/[...trpc].ts' apps/dokploy/pages/api/application.env.upsert.ts apps/dokploy/pages/api/application/env/upsert.ts apps/dokploy/server/api/routers/application.ts apps/dokploy/server/server.ts packages/server/src/db/schema/application.ts packages/server/src/services/application.ts packages/server/src/utils/env-upsert.ts apps/dokploy/__test__/env/upsert.test.ts apps/dokploy/__test__/env/application-env-upsert.test.ts
  • corepack pnpm --filter=dokploy run build-server
  • Independent Agent Flow QA reviewed the core diff and reported no blocking findings.

Not completed locally because of existing unrelated baseline/dependency issues:

  • corepack pnpm --filter=dokploy run typecheck stops on the TypeScript 6 baseUrl deprecation before project type checking.
  • Direct tsc --noEmit --ignoreDeprecations 6.0 still reports existing unrelated project type errors.
  • corepack pnpm --filter=dokploy run generate:openapi currently stops on the existing @react-email/components renderAsync export issue.

Note: I did not commit openapi.json because the repository has a post-merge Generate and Sync OpenAPI workflow that regenerates and syncs the OpenAPI artifact to Dokploy, MCP, CLI, SDK, and docs repositories after changes land on canary.

@agentHits
agentHits requested a review from Siumauricio as a code owner June 8, 2026 20:24
@dosubot dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. enhancement New feature or request labels Jun 8, 2026
@agentHits
agentHits force-pushed the feat/application-env-upsert branch from 26abf3e to 9134c9d Compare June 17, 2026 03:00
@agentHits

Copy link
Copy Markdown
Contributor Author

Rebased on latest canary (439f57566) and re-verified locally after rebase.

Checks run:

  • corepack pnpm install --frozen-lockfile
  • corepack pnpm --filter=dokploy exec vitest run --config __test__/vitest.config.ts __test__/env/upsert.test.ts __test__/env/application-env-upsert.test.ts
  • corepack pnpm exec biome check apps/dokploy/server/api/routers/application.ts packages/server/src/db/schema/application.ts packages/server/src/index.ts packages/server/src/services/application.ts packages/server/src/utils/env-upsert.ts apps/dokploy/__test__/env/upsert.test.ts apps/dokploy/__test__/env/application-env-upsert.test.ts
  • corepack pnpm --filter=dokploy run typecheck
  • corepack pnpm --filter @dokploy/server run typecheck
  • corepack pnpm --filter=dokploy run generate:openapi

CI may remain in action_required until GitHub Actions is approved for this fork PR.

@agentHits

agentHits commented Jun 17, 2026

Copy link
Copy Markdown
Contributor Author

Update: screenshot attachment is no longer applicable after the latest routing-focused update. This PR is API/server-side only; the current PR body has been updated with the relevant route and verification details.

@agentHits

Copy link
Copy Markdown
Contributor Author

@Siumauricio this PR is linked to the existing issue (#4525), was rebased on the latest canary, and was re-verified locally after rebase. Could you please review it when you have time and approve the GitHub Actions workflow run for this fork PR?

@agentHits

agentHits commented Jul 6, 2026

Copy link
Copy Markdown
Contributor Author

Update (2026-07-06): I pushed the API routing alignment into this PR after the additional validation pass.

Current confirmed shape:

  • Direct API route: POST /api/application/env/upsert.
  • Compatibility route: POST /api/application.env.upsert.
  • OpenAPI metadata path: POST /application/env/upsert.
  • The handler still performs server-side merge only, supports dryRun, expectedRevision, optional redeploy, and returns metadata without raw env values.

Related MCP PR Dokploy/mcp#56 has also been updated to call /application/env/upsert consistently in the generated OpenAPI snapshot, generated tool, and regression test.

Focused checks passed for the updated implementation: targeted env upsert Vitest, scoped Biome, build-server, diff check, and independent Agent Flow QA with no blocking findings.

Merge gate remains the same: the MCP PR should not merge until this core API lands or maintainers confirm the final API/OpenAPI contract.

Что:
- Добавил прямой Next API handler для safe application env upsert и slash/compatibility routes `/api/application/env/upsert` и `/api/application.env.upsert`.
- Подключил exact-match обработку этих routes в OpenAPI catch-all и custom HTTP server, сохранив auth, permission checks, dry-run и redeploy gating.
Зачем:
- MCP и внешние клиенты должны вызывать безопасный partial env upsert без full environment replacement и без возврата raw secret values.
Риски:
- Route selection в custom server учитывает exact-match request path и `x-forwarded-uri` для self-hosted proxy setups; auth, permissions и schema validation остаются обязательными.
Проверки:
- Команды и результаты: `git diff --cached --check` прошел; `corepack pnpm --filter=dokploy exec vitest run --config __test__/vitest.config.ts __test__/env/upsert.test.ts __test__/env/application-env-upsert.test.ts` прошел, 2 файла и 7 тестов; `CI=true corepack pnpm exec biome check ...` прошел на 10 scoped files с существующими warnings/infos; `corepack pnpm --filter=dokploy run build-server` прошел; независимый Agent Flow QA вернул pass-with-risks без блокирующих findings.
- Ограничения: full typecheck и generate:openapi в core не прошли из-за существующих unrelated project/dependency failures; live HTTP smoke через custom server не запускался.

What:
- Added a direct Next API handler for safe application env upsert and slash/compatibility routes `/api/application/env/upsert` and `/api/application.env.upsert`.
- Wired exact-match handling for those routes through the OpenAPI catch-all and custom HTTP server while preserving auth, permission checks, dry-run, and redeploy gating.
Why:
- MCP and external clients need to call safe partial env upsert without full environment replacement and without returning raw secret values.
Risks:
- Custom-server route selection considers exact-match request path and `x-forwarded-uri` for self-hosted proxy setups; auth, permissions, and schema validation remain mandatory.
Checks:
- Commands and results: `git diff --cached --check` passed; `corepack pnpm --filter=dokploy exec vitest run --config __test__/vitest.config.ts __test__/env/upsert.test.ts __test__/env/application-env-upsert.test.ts` passed, 2 files and 7 tests; `CI=true corepack pnpm exec biome check ...` passed on 10 scoped files with existing warnings/infos; `corepack pnpm --filter=dokploy run build-server` passed; independent Agent Flow QA returned pass-with-risks with no blocking findings.
- Limitations: full core typecheck and generate:openapi did not pass because of existing unrelated project/dependency failures; live HTTP smoke through the custom server was not run.
@dosubot dosubot Bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Jul 6, 2026
@agentHits

Copy link
Copy Markdown
Contributor Author

Code update pushed for the API routing part.

What changed in this PR now:

  • Added direct API support for POST /api/application/env/upsert.
  • Kept POST /api/application.env.upsert as a compatibility route.
  • Published OpenAPI metadata for POST /application/env/upsert.
  • Routed the new endpoint through exact-match handling in both the OpenAPI catch-all and the custom HTTP server.
  • Preserved auth, service permission checks, dryRun, expectedRevision, optional redeploy, and metadata-only responses without raw env values.

MCP support in Dokploy/mcp#56 has also been updated to call the slash route /application/env/upsert consistently.

Latest local checks passed: targeted Vitest env upsert tests, scoped Biome check, build-server, and independent QA review with no blocking findings.

agentHits added 3 commits July 8, 2026 11:25
Что:
- Перенесены ENV-фиксы из AgentHits-Dev в application env upsert ветку: default reads редактируют секретные env-поля, reveal endpoints возвращают raw values только после envVars read access, а write/update пути сохраняют текущие значения при redacted placeholder input.
- UI Environment теперь раскрывает реальные значения через reveal по клику на eye и использует revealed values как baseline формы.
- Добавлены focused tests для redaction helpers и reveal access guard.
Зачем:
- Пользователь должен видеть реальные ENV только по явному reveal, а MCP/API full update не должен перезаписывать сохраненные секреты строкой __DOKPLOY_REDACTED_SECRET__.
Риски:
- App/server typecheck в этой ветке остается заблокирован существующим TypeScript 6 baseline; scoped tests и lint по измененным файлам прошли.
Проверки:
- Команды и результаты: corepack pnpm --dir apps/dokploy exec vitest --config __test__/vitest.config.ts run __test__/security/env-redaction.test.ts __test__/security/service-environment.test.ts __test__/env/upsert.test.ts __test__/env/application-env-upsert.test.ts -> 4 files / 13 tests passed; corepack pnpm exec biome check <17 scoped files> -> passed; git diff --check -> passed.
- Ограничения: corepack pnpm --filter=@dokploy/server run typecheck и corepack pnpm --filter=dokploy run typecheck fail on TS5101 baseUrl deprecation with TypeScript 6.0.3; tsc --noEmit --ignoreDeprecations 6.0 still fails on existing unrelated baseline, with no filtered errors in new redaction/reveal files.

What:
- Ported ENV fixes from AgentHits-Dev into the application env upsert branch: default reads redact secret env fields, reveal endpoints return raw values only after envVars read access, and write/update paths preserve current values when redacted placeholder input is submitted.
- The Environment UI now reveals real values through reveal on eye click and uses revealed values as the form baseline.
- Added focused tests for redaction helpers and the reveal access guard.
Why:
- The user needs to see real ENV only on explicit reveal, while MCP/API full updates must not overwrite stored secrets with __DOKPLOY_REDACTED_SECRET__.
Risks:
- App/server typecheck in this branch remains blocked by the existing TypeScript 6 baseline; scoped tests and lint on changed files passed.
Checks:
- Commands and results: corepack pnpm --dir apps/dokploy exec vitest --config __test__/vitest.config.ts run __test__/security/env-redaction.test.ts __test__/security/service-environment.test.ts __test__/env/upsert.test.ts __test__/env/application-env-upsert.test.ts -> 4 files / 13 tests passed; corepack pnpm exec biome check <17 scoped files> -> passed; git diff --check -> passed.
- Limitations: corepack pnpm --filter=@dokploy/server run typecheck and corepack pnpm --filter=dokploy run typecheck fail on TS5101 baseUrl deprecation with TypeScript 6.0.3; tsc --noEmit --ignoreDeprecations 6.0 still fails on existing unrelated baseline, with no filtered errors in new redaction/reveal files.
Что:
- Добавлена маскировка credential-части customGitUrl и composeFile/env secrets в обычных compose object responses.
- create/update/delete/template compose responses теперь используют единый compose redaction wrapper.
- Добавлен тест для credential URL redaction.
Зачем:
- Исключить возврат raw compose secrets после placeholder-preserving updates.
Риски:
- Не выявлены; route-level compose coverage в этой ветке ограничена helper/focused tests.
Проверки:
- Команды и результаты: `corepack pnpm --dir apps/dokploy exec vitest --config __test__/vitest.config.ts run __test__/security/env-redaction.test.ts __test__/security/service-environment.test.ts __test__/env/upsert.test.ts __test__/env/application-env-upsert.test.ts` - passed (4 files / 14 tests); `corepack pnpm exec biome check <16 scoped files>` - passed; `git diff --check` - passed.
- Ограничения: repo typecheck не запускался повторно; ранее блокировался baseline TypeScript 6/baseUrl config.

What:
- Added masking for customGitUrl credential parts and composeFile/env secrets in normal compose object responses.
- create/update/delete/template compose responses now use one compose redaction wrapper.
- Added coverage for credential URL redaction.
Why:
- Prevent raw compose secrets from being returned after placeholder-preserving updates.
Risks:
- None identified; route-level compose coverage on this branch is limited to helper/focused tests.
Checks:
- Commands and results: `corepack pnpm --dir apps/dokploy exec vitest --config __test__/vitest.config.ts run __test__/security/env-redaction.test.ts __test__/security/service-environment.test.ts __test__/env/upsert.test.ts __test__/env/application-env-upsert.test.ts` - passed (4 files / 14 tests); `corepack pnpm exec biome check <16 scoped files>` - passed; `git diff --check` - passed.
- Limitations: repo typecheck was not rerun; it was previously blocked by baseline TypeScript 6/baseUrl config.
Что:
- Перенесен git provider redaction helper в feature-ветку.
- Compose responses теперь закрывают вложенные github/gitlab/gitea/bitbucket provider credentials вместе с env, composeFile и customGitUrl.
- Добавлен focused test на удаление provider credential fields.
Зачем:
- Закрыть reviewer blocker, где normal compose reads могли вернуть provider secrets из relation-backed responses.
Риски:
- Не выявлены.
Проверки:
- Команды и результаты: `corepack pnpm --dir apps/dokploy exec vitest --config __test__/vitest.config.ts run __test__/security/env-redaction.test.ts __test__/security/service-environment.test.ts __test__/git-provider/git-provider-access.test.ts __test__/env/upsert.test.ts __test__/env/application-env-upsert.test.ts` - passed (5 files / 41 tests); `corepack pnpm exec biome check <18 scoped files>` - passed; `git diff --check` - passed.
- Ограничения: repo typecheck не запускался повторно; ранее блокировался baseline TypeScript 6/baseUrl config.

What:
- Ported the git provider redaction helper into the feature branch.
- Compose responses now redact nested github/gitlab/gitea/bitbucket provider credentials along with env, composeFile, and customGitUrl.
- Added focused coverage for removing provider credential fields.
Why:
- Close the reviewer blocker where normal compose reads could return provider secrets from relation-backed responses.
Risks:
- None identified.
Checks:
- Commands and results: `corepack pnpm --dir apps/dokploy exec vitest --config __test__/vitest.config.ts run __test__/security/env-redaction.test.ts __test__/security/service-environment.test.ts __test__/git-provider/git-provider-access.test.ts __test__/env/upsert.test.ts __test__/env/application-env-upsert.test.ts` - passed (5 files / 41 tests); `corepack pnpm exec biome check <18 scoped files>` - passed; `git diff --check` - passed.
- Limitations: repo typecheck was not rerun; it was previously blocked by baseline TypeScript 6/baseUrl config.
@dosubot dosubot Bot added size:XXL This PR changes 1000+ lines, ignoring generated files. and removed size:XL This PR changes 500-999 lines, ignoring generated files. labels Jul 8, 2026
Что:
- Обновил feat/application-env-upsert merge-коммитом от upstream/canary без переписывания истории.
- Предварительно применил актуальные Biome-правки для command/dialog/input-group, чтобы autofix-ci не создавал конфликтный commit.
- Сохранил serverId в DeploymentJob payload для env-upsert, refresh-token и GitHub deploy producers.

Зачем:
- Чтобы pull_request autofix проходил на актуальной базе и self-hosted deploy queue сохраняла per-server partitioning.

Риски:
- Merge подтягивает большой upstream/canary payload; полный build/typecheck/test не запускались.
- Отдельных regression-тестов для tag/preview GitHub serverId нет, эти ветки проверены статически.

Проверки:
- Команды и результаты: pnpm install --frozen-lockfile - прошло; git diff --check && git diff --cached --check - прошло; corepack pnpm dlx @biomejs/biome@2.5.3 check <scoped files> - прошло; filtered tsc diagnostics check - no diagnostics; corepack pnpm --dir apps/dokploy exec vitest run targeted files - 4 files / 17 tests passed; Agent Flow reviewer recheck - pass.
- Ограничения: широкий ошибочный Vitest запуск через package script ранее упал только на real deploy tests; full pnpm typecheck/build/test не запускались.

What:
- Updated feat/application-env-upsert with an upstream/canary merge commit without rewriting history.
- Pre-applied current Biome fixes for command/dialog/input-group so autofix-ci does not need to create a conflicting commit.
- Preserved serverId in DeploymentJob payloads for env-upsert, refresh-token, and GitHub deploy producers.

Why:
- So pull_request autofix runs on the current base and the self-hosted deploy queue keeps per-server partitioning.

Risks:
- The merge pulls a large upstream/canary payload; full build/typecheck/test were not run.
- There are no dedicated regression tests for tag/preview GitHub serverId paths; those branches were checked statically.

Checks:
- Commands and results: pnpm install --frozen-lockfile - passed; git diff --check && git diff --cached --check - passed; corepack pnpm dlx @biomejs/biome@2.5.3 check <scoped files> - passed; filtered tsc diagnostics check - no diagnostics; corepack pnpm --dir apps/dokploy exec vitest run targeted files - 4 files / 17 tests passed; Agent Flow reviewer recheck - pass.
- Limitations: an earlier accidentally broad Vitest run through the package script failed only in real deploy tests; full pnpm typecheck/build/test were not run.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request size:XXL This PR changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add safe partial environment variable upsert API and MCP tool

1 participant