feat(Cubemaster): opt-in envd injection for image-based templates by 0717dn6hyc8-byte · Pull Request #626 · TencentCloud/CubeSandbox

0717dn6hyc8-byte · 2026-06-23T07:55:47Z

Summary

This PR introduces an optional envd injection path for templates created from OCI images. When enabled, CubeMaster embeds a host-side envd binary into the template root filesystem before sealing the ext4 artifact. It also records the binary's SHA256 in the template fingerprint and marks the generated template request, so restored sandboxes can start the embedded envd using an entrypoint wrapper.

By default, this injection feature is disabled to maintain the existing behavior for the community.

Motivation

Some CubeSandbox users need an envd process to be available automatically in restored sandboxes, so developer tooling can attach without requiring image authors to modify every base image manually.

What's changed

1. CLI opt-in controls

Component	Change
`cubemastercli template create-from-image`	Adds `--enable-inject-envd` to explicitly enable `envd` injection.
`cubemastercli template create-from-image`	Adds `--envd-path` to select the host-side `envd` binary.

If --envd-path is omitted, CubeMaster falls back to $CUBE_MASTER_ENVD_HOST_DIR/envd, then /usr/local/share/cubesandbox-envd/envd.

2. Template rootfs injection

Component	Behavior
`templatecenter`	Copies `envd` into `/usr/local/bin/envd` inside the exported template rootfs before sealing the ext4 artifact.
`templatecenter`	Injection only happens when `cube.master.inject_envd=true` is present in template container overrides.
`templatecenter`	Missing or unreadable `envd` binary fails the template build instead of producing an incomplete artifact.

3. Artifact fingerprinting

Component	Behavior
`templatecenter`	Includes the injected `envd` binary SHA256 in the template spec fingbled.
`templatecenter`	Leaves fingerprints unchanged for templates that do not enable `envd` injection.

This prevents reusing artifacts baked with an older envd binary after the selected envd version changes.

4. Runtime entrypoint wrapping

Component	Behavior
`CubeMaster sandbox service`	When a template carries `cube.master.inject_envd=true`, wraps the main container command .
`CubeMaster sandbox service`	Starts `/usr/local/bin/envd` in the background and then `exec`s the original command.
`CubeMaster sandbox service`	Adds the default `envd` exposed port.
`CubeMaster sandbox service`	Skips wrapping if the command is already wrapped.

Assisted-by: Claude Code:ppio/pa/gpt-5.5

cubesandboxbot · 2026-06-23T08:02:53Z

Review: PR #626 - opt-in envd injection for image-based templates

Summary

This PR is well-structured and clearly motivated. The injection is opt-in (disabled by default), the fingerprinting correctly invalidates cached artifacts when the envd binary changes, and stripPerRequestEnvdAnnotations prevents users from manually enabling envd injection on sandbox requests. Tests are comprehensive for the core injection and sidecar paths. There are a few issues worth addressing before merging.

Security

1. EnvdPath accepted via HTTP API without path validation (envd_inject.go:29-38)

CreateTemplateFromImageReq.EnvdPath is deserialized from json and passed directly to os.ReadFile with no filepath.Clean, allowlist, or symlink validation. An HTTP client could read arbitrary readable host files (credentials, TLS keys, /etc/shadow) by specifying their path: the contents would be embedded in the template rootfs artifact.

This is an opt-in flag behind a feature gate that requires explicit --enable-inject-envd CLI interaction, so the risk in practice depends on who can reach the HTTP endpoint. Still, defense-in-depth applies: normalizeTemplateImageRequest should strip or validate EnvdPath for HTTP-initiated requests, or at minimum apply filepath.Clean and reject .. components.

Additionally, EnvdPath is persisted in database job records without sanitization (unlike RegistryPassword which is zeroed before serialization in fingerprint.go:76). This should be zeroed as well.

Code Quality

2. Inconsistent opt-in sentinel between packages (envd_sidecar.go:22)

envd_sidecar.go checks val != "true" as a bare string literal, while envd_inject.go defines envdInjectAnnotationOptIn = "true" as a package-private constant. The sidecar cannot reference it. If the opt-in value ever changes, the sidecar silently stops matching. Export the constant to constants.go so both packages share the same sentinel.

3. Potential duplicate port in ExposedPorts (envd_sidecar.go:38)

injectEnvdSidecar unconditionally appends port 49983 to out.ExposedPorts before getExposedPorts runs in ConstructCubeletReq. If the template exposed-ports annotation already includes 49983, the port appears twice. Check for the port before appending, or deduplicate after both calls.

4. Error message references CLI flag in server-side code (envd_inject.go:60)

The error message mentions --envd-path but prepareEnvdInjectionPayload is also called from the HTTP API path where --envd-path is meaningless. Use a neutral message like "set CUBE_MASTER_ENVD_HOST_DIR or provide envd_path in the request".

Test Coverage

5. Notable coverage gaps in guard clauses

injectEnvdSidecar: non-cubebox instance-type early return and empty-containers guard are untested
stripPerRequestEnvdAnnotations: nil request and empty-annotations guards are untested
buildOriginalCommand: empty-command fallback returning ["sleep", "infinity"] is untested

Verdict

No blocking issues, but the EnvdPath path-validation concern (item 1) should be addressed before merge. Items 2-5 are moderate-quality improvements.

fslongjin · 2026-06-24T03:20:15Z

I want to understand the motivation behind this PR and under what scenarios it is necessary. I feel that this goes against common usage—since it's built from a container image, why would we need to additionally inject an envd? This will also affect the container's startup commands and such. I find the design of this PR very strange.

0717dn6hyc8-byte · 2026-06-24T03:47:25Z

I want to understand the motivation behind this PR and under what scenarios it is necessary. I feel that this goes against common usage—since it's built from a container image, why would we need to additionally inject an envd? This will also affect the container's startup commands and such. I find the design of this PR very strange.

For example, in development-environment / code-sandbox scenarios, we want to quickly build interactive development sandboxes from existing base images. envd provides the access layer for the development environment, but many base images do not have it preinstalled. Maintaining a large number of derived images only to integrate envd would add unnecessary cost and complexity.

Therefore, this PR injects envd during template rootfs creation. This allows those base images to gain a consistent development runtime capability without modifying their Dockerfiles or rebuilding the source images.

fslongjin · 2026-06-24T04:55:41Z

I want to understand the motivation behind this PR and under what scenarios it is necessary. I feel that this goes against common usage—since it's built from a container image, why would we need to additionally inject an envd? This will also affect the container's startup commands and such. I find the design of this PR very strange.

For example, in development-environment / code-sandbox scenarios, we want to quickly build interactive development sandboxes from existing base images. envd provides the access layer for the development environment, but many base images do not have it preinstalled. Maintaining a large number of derived images only to integrate envd would add unnecessary cost and complexity.

Therefore, this PR injects envd during template rootfs creation. This allows those base images to gain a consistent development runtime capability without modifying their Dockerfiles or rebuilding the source images.

Indeed, you make a good point. From that perspective, automatically injecting envd is indeed a great idea! Thank you for your PR. I'll study the code in your PR over the next couple of days.

kinwin-ustc · 2026-06-26T16:36:13Z

You should add your DCO;
Please modify the CubeBot review comments;
Please add corresponding documentation for the newly added CubeMasterCLI parameters.

0717dn6hyc8-byte · 2026-07-01T08:31:07Z

You should add your DCO;

Please modify the CubeBot review comments;

Please add corresponding documentation for the newly added CubeMasterCLI parameters.

Thanks for the review. I have resolved these problems:

Added DCO sign-off to all commits.
Addressed the CubeBot review comments, including envd path consistency, fingerprint error handling, redundant envd reads, misleading log wording, and the unused error return.
Added documentation for the new CubeMasterCLI parameters in both English and Chinese:
- docs/guide/tutorials/template-from-image.md
- docs/zh/guide/tutorials/template-from-image.md

Please take another look when convenient.

kinwin-ustc · 2026-07-01T11:29:48Z

There is a conflict between the current code and the main branch. You need to rebase your code based on the latest main branch and resubmit it

kinwin-ustc · 2026-07-01T12:20:33Z

 	ExposedPorts       []int32             `json:"exposed_ports,omitempty"`
 	DistributionScope  []string            `json:"distribution_scope,omitempty"`
 	ContainerOverrides *ContainerOverrides `json:"container_overrides,omitempty"`
+	EnvdPath           string              `json:"envd_path,omitempty"`


Why expose this field in the HTTP API?

kinwin-ustc · 2026-07-01T12:21:49Z


+		if strings.HasPrefix(k, "cube.container.") {
+			c.Annotations[k] = v
+		}


What is the relationship between this and envd inject?

Signed-off-by: liujie <254906391+0717dn6hyc8-byte@users.noreply.github.com>

cubesandboxbot · 2026-07-01T14:02:08Z

+func envdHostPath(req *types.CreateTemplateFromImageReq) string {
+	if req != nil && req.EnvdPath != "" {
+		return req.EnvdPath
+	}
+	dir := os.Getenv(envdHostDirEnv)
+	if dir == "" {
+		dir = envdHostDirDefault
+	}
+	return filepath.Join(dir, envdBinaryName)
+}


Security: No path validation on EnvdPath input

req.EnvdPath is accepted via the HTTP API (json:"envd_path,omitempty" on CreateTemplateFromImageReq.EnvdPath) and passed directly to os.ReadFile without filepath.Clean, allowlist checks, or symlink resolution. An HTTP client could read arbitrary readable files from the CubeMaster host (e.g., /etc/shadow, TLS keys) by specifying their path, and the content would be embedded into the template rootfs artifact.

At minimum, apply filepath.Clean and reject paths containing .. components. The canonical validation in normalizeTemplateImageRequest should strip or validate this field for HTTP-initiated requests.

cubesandboxbot · 2026-07-01T14:02:09Z

+
+func injectEnvdSidecar(ctx context.Context, req *types.CreateCubeSandboxReq, out *cubebox.RunCubeSandboxRequest) {
+	val, ok := req.Annotations[constants.CubeAnnotationsInjectEnvd]
+	if !ok || val != "true" {


Inconsistent opt-in sentinel

This check uses a bare string literal "true", while the templatecenter package defines envdInjectAnnotationOptIn = "true" as a package-private constant. If the opt-in value is ever updated (e.g., to "1" or "yes"), the sidecar will silently stop matching. Consider exporting the constant to constants.go so both packages share the same sentinel value.

cubesandboxbot · 2026-07-01T14:02:10Z

+		return
+	}
+	wrapEntrypointForEnvd(mainContainer)
+	out.ExposedPorts = append(out.ExposedPorts, envdDefaultPort)


Potential duplicate port in ExposedPorts

injectEnvdSidecar runs before getExposedPorts in ConstructCubeletReq. If the template's exposed-ports annotation already includes 49983, the port will appear twice in the final slice. Consider checking out.ExposedPorts for the port before appending, or deduplicating after both calls.

cubesandboxbot · 2026-07-01T14:02:11Z

+	srcPath := envdHostPath(req)
+	data, err := os.ReadFile(srcPath)
+	if err != nil {
+		return nil, fmt.Errorf("envd-inject: read %q (set %s or --envd-path to override): %w", srcPath, envdHostDirEnv, err)


Error message references CLI flag in server-side code

This error message mentions --envd-path, which is a CLI flag. When prepareEnvdInjectionPayload is called from the HTTP API path (not the CLI), the mention of --envd-path is confusing. Consider using a server-neutral message like "set %s or provide envd_path in the request".

0717dn6hyc8-byte requested review from chenhengqi, fslongjin, ls-ggg, tinklone and up2wing as code owners June 23, 2026 07:55