feat(helm): migrate gateway configuration to gatewayConfig

[gmenher]

Summary

Migrate the Helm chart from field-by-field gateway.toml construction to the schema-v2 gatewayConfig boundary while preserving the non-secret ConfigMap boundary and Helm-owned Secret, volume, and resource wiring.

Related Issue

Closes #3060.

Compatibility

This implementation is backwards-compatible for 0.1.x:

• Retains the deprecated sandboxRuntime, supervisor, upstreamProxy, and Kubernetes driver aliases for existing values files.
• gatewayConfig is authoritative whenever both the schema-v2 field and its legacy alias are supplied.
• Resolves aliases once and uses the same effective Kubernetes driver configuration for rendered TOML, validation, NetworkPolicy acknowledgement, and RBAC/workspace resources.
• Validates schema-v2 proxy authentication Secret references and keeps private material outside gatewayConfig.

Changes

• Add deterministic generic YAML-to-TOML rendering for the schema-v2 configuration boundary.
• Preserve chart-owned deployment inputs and derive runtime values from their resource owners.
• Cover legacy-to-schema-v2 compatibility, explicit precedence, workspace modes, NetworkPolicy acknowledgement, proxy CA wiring, and Secret-reference validation with Helm tests.
• Add parser-backed rendering and resource-coherence validation.
• Update Helm, Kubernetes, architecture, reference, and migration documentation.

Testing

• mise run ci
• mise run helm:test (170 gateway-chart tests and 5 workspace-chart tests)
• mise run helm:lint across chart overlays
• Gateway TOML parser and resource-coherence validation
• CodeRabbit review; its Secret-reference finding is covered by a regression test

Checklist

• Commits include DCO Signed-off-by trailers.
• Documentation and generated Helm README are updated.
• Compatibility and schema-v2 precedence are covered by tests.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[krishicks]

/ok to test c5afa06

[github-actions]

Label test:e2e applied, but pull-request/3384 is at c5afa06 while the PR head is cb8ae8b. A maintainer needs to comment /ok to test cb8ae8bba78baed32b09240a49abbcf7f5509c92 to refresh the mirror. Once the mirror catches up, re-run Branch E2E Checks from the Actions tab.

[krishicks]

/ok to test cb8ae8b

[gmenher]

@krishicks I pushed a new commit solving a conflict with e2e/rust/e2e-kubernetes.sh that was blocking the merge.

[krishicks]

/ok to test 5efe3e8

[krishicks]

/ok to test cfae1b6

[krishicks]

I added this to to 0.1.1 milestone as we're freezing what goes into 0.1.0. For this to actually land in 0.1.1 it would need to be implemented in a backwards-compatible way. Failing that this would need to be pushed to 0.2.0 which is the next release where breaking changes can get in.

[gmenher]

Rebased onto latest main and conflicts are resolved. @krishicks could I have the test suite re-executed with /ok to test when convenient? Thanks!

[dvavili]

I did some analysis as the goal is to have this rolled in a backwards compatible manner. Posting my analysis:

Compat check against 0.1.x: legacy server.oidc.* without role names loses RBAC on upgrade:

I rendered the chart at the merge-base (b8ffe52) and at this PR's head (de1451b) with the same legacy ci/values-*.yaml overlays, then compared the parsed gateway.toml and every other resource. All 18 overlays render on both. Nothing is removed or changed except the case below. The rest are additions that match gateway defaults (disable_tls = false, empty otlp.service_name, empty scopes_claim).

Regression: a values file that sets server.oidc.issuer and audience but no role names (for example values-gateway-tls.yaml) now renders:

[openshell.gateway.oidc]
admin_role = ""
user_role = ""
roles_claim = ""

The old template emitted roles_claim, admin_role and user_role only when non-empty (gateway-config.yaml L163-171), so the gateway fell back to realm_access.roles / openshell-admin / openshell-user.

With explicit empty strings:

Config-file values replace defaulted CLI args (cli.rs L1171-1178), so the defaults are lost.
Both roles empty puts AuthzPolicy in authentication-only mode, where any valid token is authorized (auth/authz.rs L20-26, L83-85).
So an existing install that was enforcing the default roles would accept any valid token from its issuer after upgrading. That is a silent loosening of an access control, which conflicts with the "backwards-compatible for 0.1.x" claim.

Repro:

helm template -t deploy/helm/openshell -f <base>/deploy/helm/openshell/ci/values-gateway-tls.yaml
# compare [openshell.gateway.oidc] between merge-base and this PR

Suggested fix: in the legacy OIDC translation, add roles_claim, admin_role and user_role only when the legacy value is non-empty, as the old template did and as this PR already does for the other optional fields. Please also add a Helm test: legacy server.oidc.issuer only → gateway.toml has no admin_role, user_role or roles_claim. Another test could pin that an explicit schema-v2 admin_role = "" is still honoured.