Propagate proxy env to engine-generic workers via engine-cm#393
Merged
Conversation
The engine spawns engine-generic worker pods (as a Deployment) by code, and those pods source their env from the engine-cm ConfigMap via envFrom (deployment_manager.py), not from the engine container's own env. So add HTTP(S)_PROXY/NO_PROXY to engine-cm: one edit reaches the engine main container, engine-generic workers, engine-orchestrator and the copy-deps job (all envFrom: engine-cm), and the in-pod os.fork() children inherit it. Reuses the existing engine proxy values (GetEngineProxyEnv); no installer or engine-code change needed. The pippin init container keeps its explicit proxy env since it has no envFrom. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
moshe-kabala
approved these changes
Jun 28, 2026
| GENERIC_CALCULATOR_ENTRY_POINT: {{ .Values.entry_point }} | ||
| GENERIC_ACTIVATE_NFS: "{{ .Values.metrics_activate_nfs }}" | ||
| GENERIC_HOST_PATH: {{ .Values.localDataDirectories | join ":" }} | ||
| {{- if .Values.http_proxy }} |
Contributor
There was a problem hiding this comment.
one is http ant the second is https
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #392 (which covered the pippin push-job init container). This extends proxy propagation to the engine-generic worker pods and the rest of the engine workloads.
Why
The engine process spawns engine-generic worker pods itself (as a Deployment,
create_namespaced_deploymentin the engine repo'sdeployment_manager.py), and those pods source their env from theengine-cmConfigMap viaenvFrom— not from the engine container's own env. So putting proxy vars on the engine container would not reach the workers; the correct single source isengine-cm.What
Add
HTTP_PROXY/HTTPS_PROXY/NO_PROXY(upper + lower case) toengine-cm, rendered only when set. One edit reaches everyenvFrom: engine-cmconsumer:In-pod
os.fork()children inherit the pod env automatically. Reuses the existing engine proxy values (GetEngineProxyEnv→.Values.http_proxy/https_proxy/no_proxy) introduced in #392 — no installer change and no engine-code change (the worker Deployment referencesengine-cmby name, so it picks up the keys at runtime). The pippin init container keeps its explicit proxy env (it has noenvFrom).NO_PROXYis already augmented with in-cluster targets (zot registry, minio,.svc,.cluster.local,10.42/10.43CIDRs) so workers' in-cluster traffic bypasses the proxy.Validation
helm template:engine-cmhas 0 proxy keys by default; all 6 keys render (inside theengine-cmConfigMap) when proxy values are set.go build,gofmt, and fullgo test ./...all pass.tensorleap1.6.35→1.6.36,tensorleap-engine1.0.615→1.0.616.🤖 Generated with Claude Code