Hive Commons (CNCF Sandbox) integrated OpenShell as a coding-agent sandbox runtime — measurements, friction, two asks

[clubanderson]

Hi OpenShell folks — Andy from Hive Commons (CNCF Sandbox, https://github.com/hivecommons/hive). We've shipped OpenShell as an opt-in sandbox runtime for the coding agents hive runs on Kubernetes, and just finished measuring it on a Linode LKE cluster (Debian 13, kernel 6.12, Calico). Sharing what we found in case it's useful, plus a couple of questions.

What we built: sandbox.runtime: openshell next to our job and podman runtimes — ADR: https://github.com/hivecommons/hive/blob/v5/src/docs/adr/0026-sandbox-runtime-selection.md · operator guide: https://github.com/hivecommons/hive/blob/v5/src/deploy/openshell.md · PRs hivecommons/hive#17638, #17646; prototype results hivecommons/hive#17643. Hive derives the egress policy from its own allowlist (github.com, api.github.com) per binary, mounts the agent workspace from a shared RWX claim via caller driver_config, and deletes the sandbox when the kick ends.

Numbers (gateway/supervisor/sandbox 0.1.8, same node, median):

	plain K8s Job	OpenShell	ratio
cold start → shell ready → result	6.0 s	10.0 s	1.67×
warm start	3.8 s	9.1 s	2.4×
memory per sandbox	~0	+38 Mi (supervisor 18 + workload 20)
per-node images		+4.9 MB sandbox, +26 MB supervisor; gateway 58 MB once

Egress enforcement was exactly right: allowed hosts 200/OK, everything else refused in ~2 ms; non-root, CapEff 0, seccomp on. Nice work — it "just enforced".

Friction we hit (all worked around, documented on our side):

• Chart defaults gave us 27 s warm starts; the 20 s was provision+attach of the per-sandbox workspace-* block PVC. server.workspaceStorageClass on NFS fixed it.
• MAX_ROUTABLE_NAME_LEN = 19 and the reserved workspace volume name are only discoverable from the gateway's error; allowed_pod_labels/allowed_pod_annotations default-deny likewise.
• The gateway forces runAsUser 10001 over the image's USER (fine, but surprised us on a shared claim).
• alpine images fail every time with spawn sandbox entrypoint process '/bin/sh': Permission denied (os error 13); glibc images are fine.
• 1 of 21 runs: supervisor session closed right after ReportSandboxConfiguration and never reconnected, so the sandbox never became Ready.
• Preflight lesson from another cluster: a kernel ≥ 6.2 is not enough — landlock must be in the active /sys/kernel/security/lsm list (OL9 UEK8 6.12 had it compiled but not active).

Asks: (1) Is there a way to skip the per-sandbox workspace PVC when the caller mounts its own volume, or make it an emptyDir? (2) For machine clients like hive's controller, is a static/service-account credential planned, or is OIDC JWT / mTLS the intended path? Also happy to open an issue for the alpine /bin/sh EACCES if you'd like a repro.

We'll publish per-runtime results on https://hivecommons.dev/benchmarks/ and would be glad to collaborate on the benchmark harness or docs. Thanks!