Skip to content

Commit 85d4fa2

Browse files
committed
fix(docker): document CDI context mount
Signed-off-by: Evan Lezar <elezar@nvidia.com>
1 parent 991c43a commit 85d4fa2

4 files changed

Lines changed: 21 additions & 16 deletions

File tree

‎architecture/compute-runtimes.md‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -245,10 +245,11 @@ For all in-tree drivers, this is equivalent to selecting a single GPU.
245245
For Docker GPU sandboxes, the driver treats CDI specs as runtime metadata for
246246
both outer injection and inner sandbox policy. It selects opaque CDI device IDs,
247247
passes them to Docker, mounts daemon-reported CDI spec directories into
248-
supervisor-only paths, and uploads a versioned CDI context before starting the
249-
container. The supervisor resolves that context inside the sandbox and derives
250-
Landlock paths and supplemental groups from CDI `containerEdits`. Host-side CDI
251-
spec paths are diagnostic only and are never treated as sandbox policy paths.
248+
supervisor-only paths, and bind-mounts a gateway-owned versioned CDI context
249+
read-only before creating the container. The supervisor resolves that context
250+
inside the sandbox and derives Landlock paths and supplemental groups from CDI
251+
`containerEdits`. Host-side CDI spec paths are diagnostic only and are never
252+
treated as sandbox policy paths.
252253
Kubernetes must not infer CDI device IDs from the `nvidia.com/gpu` resource
253254
request; it needs a node-local selected-device handoff before using the same
254255
supervisor resolver.

‎crates/openshell-driver-docker/Cargo.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,13 +38,13 @@ miette = { workspace = true }
3838
toml = { workspace = true }
3939
tower-http = { workspace = true }
4040
http = { workspace = true }
41-
tar = "0.4"
4241

4342
[dev-dependencies]
4443
openshell-otel-test-support = { path = "../openshell-otel-test-support" }
4544
opentelemetry = { workspace = true }
4645
opentelemetry_sdk = { workspace = true, features = ["testing"] }
4746
prost-types = { workspace = true }
47+
tar = "0.4"
4848
temp-env = "0.3"
4949
tempfile = "3"
5050
tracing-subscriber = { workspace = true }

‎crates/openshell-driver-docker/README.md‎

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -107,7 +107,7 @@ contract:
107107
| `PidsLimit` | Enforces the sandbox PID budget at the Docker cgroup layer. Set `[openshell.drivers.docker].sandbox_pids_limit = 0` to inherit the Docker/runtime default. |
108108
| CDI GPU request | Uses opaque `driver_config.cdi_devices` values when set; otherwise selects the requested count of NVIDIA CDI GPUs in round-robin order when daemon CDI support is detected. Docker daemon `/info` can permit `nvidia.com/gpu=all` as a WSL2 all-only compatibility fallback, where it counts as one selectable device. Exact CDI device lists must not contain duplicates and must match the effective GPU count. |
109109
| `policy-dns-transparent-tcp` capability | Declares that the combined Docker supervisor can own namespace-local DNS/TCP capture and coupled workload restart. The shared supervisor still owns DNS eligibility, mappings, authorization, pinned dialing, relaying, and OCSF decisions. The marker is stripped from the workload environment. |
110-
| CDI context upload | For GPU/CDI sandboxes only, mounts daemon-reported CDI spec directories read-only under `/run/openshell/supervisor/cdi-specs/<n>` and uploads `/run/openshell/supervisor/cdi-context.json` after container create and before start. |
110+
| CDI context mount | For GPU/CDI sandboxes only, creates a gateway-owned context file and bind-mounts it read-only at `/run/openshell/supervisor/cdi-context.json`; daemon-reported CDI spec directories are mounted read-only under `/run/openshell/supervisor/cdi-specs/<n>`. |
111111

112112
The agent child process does not retain these supervisor privileges.
113113

@@ -118,11 +118,13 @@ CDI device IDs from `driver_config.cdi_devices` or the daemon's discovered CDI
118118
inventory, then passes the same IDs to Docker with a CDI `DeviceRequest`.
119119

120120
When a GPU/CDI request is present, the driver also mounts the Docker
121-
daemon-reported `Info.CDISpecDirs` into supervisor-only paths and uploads a
122-
small versioned CDI context through Docker's container archive API. The context
123-
uses container-side spec paths for resolution and keeps host-side spec sources
124-
diagnostic-only. If the upload fails, the driver removes the created container
125-
and sandbox token file before reporting the failure.
121+
daemon-reported `Info.CDISpecDirs` into supervisor-only paths. Before container
122+
creation, it writes a small versioned CDI context in gateway-owned state and
123+
bind-mounts it read-only into the supervisor. The context uses container-side
124+
spec paths for resolution and keeps host-side spec sources diagnostic-only. If
125+
context or token creation fails, the driver removes any created state files; if
126+
container creation or start fails, it also removes the container and state
127+
files before reporting the failure.
126128

127129
The sandbox supervisor resolves the selected IDs from those mounted specs
128130
before it launches agent processes. CDI device nodes become read-write

‎docs/reference/sandbox-compute-drivers.mdx‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -170,11 +170,13 @@ For GPU-backed Docker sandboxes, configure Docker CDI before starting the gatewa
170170
For Docker GPU/CDI sandboxes, OpenShell uses Docker's selected CDI device IDs
171171
and daemon-reported CDI spec directories to build a supervisor-only CDI
172172
context. The driver mounts the spec directories read-only into the sandbox
173-
container, uploads `cdi-context.json` before starting the container, and removes
174-
the created container if that upload fails. The supervisor resolves the context
175-
inside the sandbox and derives the inner filesystem and supplemental group
176-
requirements from CDI specs. Non-GPU Docker sandboxes do not receive the CDI
177-
context, spec mounts, or CDI-derived policy changes.
173+
container. Before creation, it writes a gateway-owned `cdi-context.json` and
174+
bind-mounts it read-only into the supervisor. If context or token creation
175+
fails, the driver removes the created state files; if container creation or
176+
start fails, it also removes the container and state files. The supervisor
177+
resolves the context inside the sandbox and derives the inner filesystem and
178+
supplemental group requirements from CDI specs. Non-GPU Docker sandboxes do not
179+
receive the CDI context, spec mounts, or CDI-derived policy changes.
178180

179181
### Docker Driver Config Mounts
180182

0 commit comments

Comments
 (0)