fix(ci)!: remove gateway callback listener dependency (#3365)

* fix(ci): repair post-merge release canary

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(packaging): bootstrap canary runtime prerequisites

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* ci(canary): collect macOS VM diagnostics

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* ci(canary): pin libkrun-compatible macOS runner

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* ci(canary): limit macOS smoke test to package startup

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* refactor(compute)!: remove gateway callback listeners

Run Docker supervisors on host networking so they use the operator-configured primary gateway endpoint. Remove the unused compute-driver callback listener negotiation and listener-scoped routing machinery.

BREAKING CHANGE: The ComputeDriver API no longer exposes GetGatewayListenerRequirements or GatewayListenerRequirement. External drivers must regenerate bindings and connect supervisors to the configured primary gateway endpoint.

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* refactor(docker): use sandbox runtime image in launcher

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* test(podman): exercise production endpoint selection

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(e2e): route supervisors to reachable gateways

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* test(e2e): align Podman endpoint fixtures

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(docker): preserve host aliases for supervisors

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(docker): align sandbox host gateway pin

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(e2e): address Docker fixtures by bridge IP

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* test(e2e): serialize sandbox lifecycle cases

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(e2e): host Docker TCP fixture with gateway

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

* fix(e2e): use loopback for host-network supervisor

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

---------

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

Author: drew

.agents/skills/test-release-canary/SKILL.md

@@ -112,6 +112,7 @@ helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart \
   --namespace openshell --create-namespace \
   --set server.disableTls=true \
   --set server.telemetryEnabled=false \
+  --set supervisor.sandboxRuntime.networkPolicyEnforced=true \
   --wait --timeout 5m
 
 kubectl wait --namespace openshell \

.github/workflows/release-canary.yml

@@ -35,15 +35,28 @@ jobs:
           launchctl setenv OPENSHELL_COMPUTE_DRIVER vm
           launchctl setenv OPENSHELL_TELEMETRY_ENABLED "$OPENSHELL_TELEMETRY_ENABLED"
 
-      - name: Install and check status
+      - name: Install and check gateway status
         run: |
           set -euo pipefail
           curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/${{ github.event.workflow_run.head_sha || github.sha }}/install.sh | sh
+          openshell --version
           openshell status
-          sandbox="release-canary-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}"
-          openshell sandbox create --name "$sandbox" --detach
-          openshell sandbox exec --name "$sandbox" --no-tty -- true
-          openshell sandbox delete "$sandbox"
+
+      # GitHub-hosted macOS runners do not expose the Hypervisor.framework
+      # support libkrun needs. Sandbox launch is covered by the VM E2E lane.
+      - name: Collect Homebrew diagnostics
+        if: failure()
+        run: |
+          set +e
+          brew services info openshell
+          for log in \
+            "$(brew --prefix)/var/log/openshell/openshell-gateway.out.log" \
+            "$(brew --prefix)/var/log/openshell/openshell-gateway.err.log"; do
+            if [ -f "$log" ]; then
+              echo "--- $log ---"
+              tail -n 300 "$log"
+            fi
+          done
 
   ubuntu:
     name: Ubuntu Docker
@@ -68,7 +81,7 @@ jobs:
           set -euo pipefail
           curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/${{ github.event.workflow_run.head_sha || github.sha }}/install.sh | sh
           openshell status
-          sandbox="release-canary-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}"
+          sandbox="rc-${GITHUB_RUN_ID}"
           openshell sandbox create --name "$sandbox" --detach
           openshell sandbox exec --name "$sandbox" --no-tty -- true
           openshell sandbox delete "$sandbox"
@@ -153,7 +166,7 @@ jobs:
             DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/0/bus \
             OPENSHELL_VERSION="$OPENSHELL_VERSION" \
             OPENSHELL_TELEMETRY_ENABLED="$OPENSHELL_TELEMETRY_ENABLED" \
-            CANARY_SANDBOX="release-canary-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}" \
+            CANARY_SANDBOX="rc-${GITHUB_RUN_ID}" \
             INSTALL_SH_URL="https://raw.githubusercontent.com/NVIDIA/OpenShell/${{ github.event.workflow_run.head_sha || github.sha }}/install.sh" \
             bash -s <<'EOF'
           set -euo pipefail
@@ -235,7 +248,7 @@ jobs:
       - name: Create and exercise a sandbox
         run: |
           set -euo pipefail
-          sandbox="release-canary-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}"
+          sandbox="rc-${GITHUB_RUN_ID}"
           openshell sandbox create --name "$sandbox" --detach
           openshell sandbox exec --name "$sandbox" --no-tty -- true
           openshell sandbox delete "$sandbox"
@@ -292,7 +305,9 @@ jobs:
             --version 0.0.0-dev \
             --namespace "$RELEASE_NAMESPACE" --create-namespace \
             --set server.disableTls=true \
+            --set server.auth.allowUnauthenticatedUsers=true \
             --set "server.telemetryEnabled=${OPENSHELL_TELEMETRY_ENABLED}" \
+            --set supervisor.sandboxRuntime.networkPolicyEnforced=true \
             --wait --timeout 5m
 
       - name: Verify gateway pod is Ready
@@ -333,7 +348,7 @@ jobs:
           set -euo pipefail
           openshell gateway add http://127.0.0.1:8080 --local --name "$KIND_GATEWAY_NAME"
           openshell status
-          sandbox="release-canary-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}"
+          sandbox="rc-${GITHUB_RUN_ID}"
           openshell sandbox create --name "$sandbox" --detach
           openshell sandbox exec --name "$sandbox" --no-tty -- true
           openshell sandbox delete "$sandbox"

architecture/compute-runtimes.md

@@ -51,8 +51,9 @@ creation. Drivers that enforce policy outside the standard supervisor fetch
 later revisions through `GetSandboxConfig` and acknowledge them through
 `ReportPolicyStatus`.
 Process-identity omissions are preserved across this boundary so every driver
-can apply its native image or runtime defaults. Driver-requested listeners are
-structurally validated and remain restricted to sandbox callback RPCs.
+can apply its native image or runtime defaults. Drivers connect supervisors to
+the operator-configured gateway endpoint; they do not request additional
+gateway listeners.
 
 Canonical main-process support is part of the `ComputeDriver` contract. Every
 in-tree and extension driver must forward the exact specification; it is not an

architecture/gateway.md

@@ -6,7 +6,7 @@ attachments; and asks compute runtimes to create or delete sandbox workloads.
 
 ## Responsibilities
 
-- Authenticate clients and sandbox callbacks.
+- Authenticate clients and sandbox supervisor sessions.
 - Serve gRPC APIs for sandbox lifecycle, provider management, policy updates,
   settings, logs, watch streams, and relay forwarding.
 - Serve HTTP endpoints for health, WebSocket tunnels, and edge-auth flows.
@@ -49,7 +49,7 @@ versions fail before runtime construction, and driver settings belong only to
 Package lifecycle code may replace an exact package-generated v1 default, but
 it preserves edited configurations for explicit operator migration.
 
-Gateway listener TLS and sandbox callback TLS are separate inputs. A selected
+Gateway listener TLS and sandbox supervisor TLS are separate inputs. A selected
 local Docker, Podman, or VM driver requires a complete guest bundle whenever
 the gateway listener uses TLS; package-managed local TLS can supply that bundle.
 Kubernetes instead projects guest credentials through its configured Secret.
@@ -141,24 +141,16 @@ template, provider, and service collection list RPCs use the same field with an
 all-workspaces marker. Platform-global policy operations omit both `sandbox`
 and `workspace_scope`, while sandbox policy operations require both.
 
-Docker and Podman report the local address through which their sandboxes can
-reach the gateway. When the primary listener covers that address, the gateway
-reuses it; sandbox JWT authentication and its RPC allowlist remain the callback
-authorization boundary. When the primary listener does not cover the address,
-the gateway adds a callback-only listener. Additional callback listeners accept
-only gRPC methods classified as sandbox-callable by the gateway's generated
-authorization metadata. They reject user and administrator APIs, health,
-reflection, and HTTP routes before normal request
-authentication. The operator-configured primary listener retains the full
-multiplexed API surface.
-
-The `rpc_auth` classification is also the source of truth for negotiated
-listener exposure: marking an RPC as `sandbox` or `dual` makes it callable on
-these listeners. Review such changes as both authorization and network-surface
-changes. Listener requirements are currently authorized only for the built-in
-Docker and Podman drivers. Operator-granted listener capabilities for external
-drivers are tracked in
-[#2539](https://github.com/NVIDIA/OpenShell/issues/2539).
+Docker and Podman supervisors use host networking and connect through the
+gateway's primary listener. On Linux, local supervisors use the primary
+loopback endpoint. Sandbox JWT authentication and the generated sandbox RPC
+allowlist remain the authorization boundary; the gateway does not negotiate or
+bind compute-driver-specific listeners.
+
+The `rpc_auth` classification is the source of truth for supervisor access.
+Marking an RPC as `sandbox` or `dual` makes it callable by an authenticated
+sandbox principal on the primary listener. Review such changes as
+authorization-surface changes.
 
 Operators can configure a gateway-wide gRPC request rate limit. The limit is
 applied only to gRPC API traffic after protocol multiplexing; health, metrics,
@@ -1059,25 +1051,16 @@ system entry instead of pretending to delete package-manager owned state.
 
 - Gateway TLS and client certificate distribution are deployment concerns owned
   by the operator or packaging layer.
-- Compute runtimes own the mechanics of starting workloads and injecting
-  callback configuration. Local Docker, Podman, and VM callback endpoints can
-  be derived from their fixed host aliases. Kubernetes requires an explicit
-  endpoint from driver placement; Helm renders it from the gateway Service
-  name and namespace rather than inferring it from sandbox placement.
-- Docker-backed local gateways use Docker's `host-gateway` callback alias on
-  macOS and Docker Desktop-style runtimes. They request IPv4 loopback callback
-  reachability and add a listener only when the primary does not cover it.
-  Native Linux Docker may expose an additional bridge-gateway listener because
-  the host can bind that bridge IP.
-- Podman-backed macOS gateways use gvproxy's host-loopback IP for sandbox host
-  aliases by default so stale Podman machine images do not need Podman's
-  `host-gateway` resolver. Linux Podman keeps the resolver unless
-  `host_gateway_ip` is configured. Rootful Podman can request its exact bridge
-  gateway listener. Rootless Podman explicitly reporting pasta requests the
-  private IPv4 source selected by the host default route rather than an
-  arbitrary private interface. Slirp4netns, other helpers, and missing helper
-  metadata fail closed for local callbacks until a rootless-network namespace
-  relay is available.
+- Compute runtimes own the mechanics of starting workloads and injecting the
+  gateway endpoint. Docker and Podman supervisors use host networking; local
+  Linux supervisors use the gateway's primary loopback endpoint. Kubernetes
+  uses the gateway Service rendered by Helm. VM supervisors use their
+  runtime-specific host route.
+- Docker Desktop requires host networking to be enabled and cannot combine it
+  with Enhanced Container Isolation. Set an explicit remote `grpc_endpoint`
+  when the gateway is not reachable on the Docker daemon host.
+- Podman Machine uses gvproxy's host-loopback route on macOS. Native Linux
+  Podman uses the primary loopback endpoint.
 - Gateway restarts recover persisted objects from storage, but live relay
   streams must be re-established by supervisors.
 - User-facing behavior changes must update published docs in `docs/`; this file

architecture/sandbox.md

@@ -400,7 +400,7 @@ The canonical main process receives the declared workload environment before
 supervisor-only values are stripped and provider placeholders are injected.
 Template environment is treated like user-provided sandbox environment. It can
 shape the workload child, but it cannot override driver-controlled identity,
-gateway callback, TLS, relay socket, proxy, provider, or supervisor coordination
+gateway endpoint, TLS, relay socket, proxy, provider, or supervisor coordination
 variables. Drivers and the supervisor rewrite those reserved values after image
 and template environment are considered.
 

crates/openshell-core/src/driver_utils.rs

@@ -7,58 +7,6 @@ use std::path::{Path, PathBuf};
 
 use crate::proto::compute::v1::DriverSandbox;
 
-/// Built-in sandbox network callback routes used to derive a callback endpoint
-/// when an operator does not configure a per-driver `grpc_endpoint` override.
-#[derive(Debug, Clone, Copy, PartialEq, Eq)]
-pub enum GatewayCallbackRoute {
-    /// A Docker container reaches the host through Docker's gateway alias.
-    Docker,
-    /// A Podman container reaches the host through Podman's gateway alias.
-    Podman,
-    /// A libkrun guest reaches the host through gvproxy's gateway alias.
-    Vm,
-}
-
-/// Build the endpoint a sandbox uses to call its gateway for a known route.
-///
-/// The result is deliberately derived by the gateway rather than baked into
-/// individual driver defaults. A configured `grpc_endpoint` remains an
-/// operator override for remote or non-standard deployments.
-#[must_use]
-pub fn gateway_callback_endpoint(
-    route: GatewayCallbackRoute,
-    gateway_port: u16,
-    gateway_tls_enabled: bool,
-) -> String {
-    let scheme = if gateway_tls_enabled { "https" } else { "http" };
-    let host = match route {
-        GatewayCallbackRoute::Docker | GatewayCallbackRoute::Vm => "host.openshell.internal",
-        GatewayCallbackRoute::Podman => "host.containers.internal",
-    };
-    format!("{scheme}://{host}:{gateway_port}")
-}
-
-#[cfg(test)]
-mod callback_endpoint_tests {
-    use super::{GatewayCallbackRoute, gateway_callback_endpoint};
-
-    #[test]
-    fn derives_endpoint_for_each_builtin_route() {
-        assert_eq!(
-            gateway_callback_endpoint(GatewayCallbackRoute::Docker, 17670, false),
-            "http://host.openshell.internal:17670"
-        );
-        assert_eq!(
-            gateway_callback_endpoint(GatewayCallbackRoute::Podman, 17670, true),
-            "https://host.containers.internal:17670"
-        );
-        assert_eq!(
-            gateway_callback_endpoint(GatewayCallbackRoute::Vm, 17670, true),
-            "https://host.openshell.internal:17670"
-        );
-    }
-}
-
 // ---------------------------------------------------------------------------
 // Sandbox container/pod label keys (openshell.ai/ namespace)
 // ---------------------------------------------------------------------------

crates/openshell-driver-docker/README.md

@@ -35,9 +35,9 @@ mediates every supported TCP and DNS operation, attributes it to the calling
 binary, and sends the request across the private channel. The supervisor
 authorizes the request before it opens an upstream connection. Docker's absent
 workload network is the mandatory outer fence if mediation fails or is
-bypassed. Only the trusted supervisor companion joins the driver-owned bridge,
-where it originates approved egress and can resolve other services on that
-network.
+bypassed. The trusted supervisor companion uses Docker host networking, where
+it reaches the gateway's primary loopback listener and originates approved
+egress.
 
 The driver copies trusted runtime bytes from the configured supervisor image
 through the Docker archive API. No workload launch depends on a host bind
@@ -73,7 +73,7 @@ LSM decisions remain authoritative.
 | `cap_drop = ALL`, no `cap_add`, no-new-privileges | Prevents either container from acquiring Linux capabilities. |
 | Docker default seccomp and AppArmor profiles | Retains runtime hardening; startup confirmation fails closed if nested seccomp notification is unavailable. |
 | `network_mode = none` on the workload | Removes direct external routes. |
-| Driver-owned bridge on the supervisor | Lets the trusted supervisor originate approved gateway and upstream connections and use Docker service discovery. |
+| `network_mode = host` on the supervisor | Lets the trusted supervisor reach the gateway's primary loopback listener and originate approved upstream connections. |
 | `restart_policy = no` | Keeps canonical main-process exit terminal. |
 | `PidsLimit` | Applies the configured sandbox PID budget. Omit `sandbox_pids_limit` to use OpenShell's default. Explicit zero is invalid. |
 | Private named volumes | One carries the authenticated sandbox/supervisor channel. The other is mounted only into the supervisor and contains its JWT and private gateway credentials. |
@@ -128,22 +128,25 @@ dynamically linked glibc `/openshell-supervisor` binary that runs in the
 host-networked supervisor container. Release and gateway image builds bake
 matching image tags into the binary.
 
-## Callback and TLS
+## Gateway session and TLS
 
 `OPENSHELL_ENDPOINT` and gateway authentication material are injected only into
 the supervisor companion. The workload never receives the sandbox JWT, gateway
 client TLS key, policy authority, or interception CA private key.
 
-When no endpoint is configured, the driver derives
-`host.openshell.internal:<gateway-port>`. Native Linux uses the managed bridge
-gateway. Docker Desktop and compatible VM-backed daemons use Docker's
-`host-gateway` route. A configured HTTPS server certificate must include the
+When no endpoint is configured, the supervisor connects to
+`127.0.0.1:<gateway-port>`. Set `grpc_endpoint` when the gateway is not on the
+Docker daemon host. A configured HTTPS server certificate must include the
 endpoint host in its subject alternative names.
 
-The driver pins a concrete managed-bridge address in the sandbox descriptor.
-For Docker's special `host-gateway` route, it leaves the address unpinned so
-the supervisor resolves the driver-injected alias in its own container. This
-avoids treating the Docker VM's loopback as the desktop host.
+The driver publishes host loopback as the backend address for
+`host.openshell.internal`. Policy DNS resolves that reserved name through the
+mediated path, so policies can reach host services without a Docker bridge,
+container DNS alias, or another gateway listener.
+
+Docker Engine on Linux supports host networking directly. Docker Desktop
+requires host networking to be enabled in Settings and does not support it
+when Enhanced Container Isolation is enabled.
 
 The supervisor owns these security-critical variables: