Skip to content

docs(proposal): add dynamic environment variable injection design proposal#597

Open
furykerry wants to merge 1 commit into
openkruise:masterfrom
furykerry:proposal/dynamic-env-injection
Open

docs(proposal): add dynamic environment variable injection design proposal#597
furykerry wants to merge 1 commit into
openkruise:masterfrom
furykerry:proposal/dynamic-env-injection

Conversation

@furykerry

Copy link
Copy Markdown
Member

Ⅰ. Describe what this PR does

Add a design proposal for per-claim environment variable injection into pre-warmed sandbox pods.

The proposal introduces run_with_envs.sh — a deferred execution wrapper that blocks the container's main command until the sandbox is claimed and env vars are written. Key design points:

  • EnvVarsInjectionPolicy field on SandboxClaim and SandboxSpec: None (default) or Auto (opt-in). No annotation-based toggle.
  • Command + exec-probe wrapping: Under Auto, the controller wraps the container command and exec probes (startupProbe, livenessProbe, readinessProbe) with run_with_envs.sh. Probe mode (--probe) returns success immediately before env initialization, keeping pre-warmed pods healthy.
  • Annotation-only env persistence: Env vars are written to AnnotationInitRuntimeRequest only — not to the container spec — to avoid triggering the in-place upgrade path which does not support env var changes.
  • Resume/upgrade restoration: The Initialize() hook re-writes env vars to the filesystem after pod recreation.
  • Limitations: postStartHook and non-exec probes (httpGet, tcpSocket) are not wrapped. Secrets should use traffic-extension SecurityProfile instead of envVars.

Ⅱ. Does this pull request fix one issue?

NONE

Ⅲ. Describe how to verify it

  1. Review the proposal document at docs/proposals/20260701-dynamic-env-injection.md
  2. Confirm the design aligns with existing sandbox lifecycle flows (claim, pause/resume, upgrade)

Ⅳ. Special notes for reviews

  • The EnvVarsInjectionPolicy default is None to preserve backward compatibility — Auto is opt-in.
  • The postStartHook and non-exec probe limitations are explicit user-facing constraints.
  • In-place upgrade support for env vars is tracked as future work.

…posal

Introduce a design proposal for per-claim environment variable injection
into pre-warmed sandboxes via run_with_envs.sh wrapper.

Key design decisions:
- EnvVarsInjectionPolicy field (None default, Auto opt-in) on
  SandboxClaim and Sandbox CR spec controls injection behavior
- run_with_envs.sh wraps container command and exec probes; supports
  probe mode (--probe) returning success before env initialization
- Env vars persisted to AnnotationInitRuntimeRequest annotation only
  (not container spec) to avoid triggering in-place upgrade
- postStartHook and non-exec probes (httpGet/tcpSocket) are not wrapped
  (documented limitations)
- Secrets excluded from envVars; use traffic-extension SecurityProfile
- Environment restoration after resume/upgrade via Initialize() hook

Signed-off-by: 守辰 <shouchen.zz@alibaba-inc.com>
@kruise-bot kruise-bot requested review from AiRanthem and zmberg July 1, 2026 08:56
@kruise-bot

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from furykerry by writing /assign @furykerry in a comment. For more information see:The Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@codecov

codecov Bot commented Jul 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.70%. Comparing base (6205b42) to head (1e694bc).
⚠️ Report is 4 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #597      +/-   ##
==========================================
- Coverage   79.84%   79.70%   -0.14%     
==========================================
  Files         202      205       +3     
  Lines       14795    15282     +487     
==========================================
+ Hits        11813    12181     +368     
- Misses       2552     2658     +106     
- Partials      430      443      +13     
Flag Coverage Δ
unittests 79.70% <ø> (-0.14%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@zmberg zmberg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded — see the inline comments.

- Secret/ConfigMap-based env var sources (future enhancement).
- Env var injection without agent-runtime (envd-less path).
- In-place upgrade support for env var changes on the Sandbox CR. Currently, modifying the
container spec triggers in-place upgrade which does not support env var updates.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add VM-based sandboxes (e.g. OSWorld) as a non-goal too. They have no container concept, so the run_with_envs.sh command-wrapping approach fundamentally does not apply.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants