Skip to content

Commit 9ac0042

Browse files
pimlockpimlock
committed
docs(middleware): clarify navigation and service contracts
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
1 parent afa9f44 commit 9ac0042

8 files changed

Lines changed: 76 additions & 75 deletions

File tree

‎architecture/sandbox-limits.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,7 @@ include 64 KiB service config, 4 KiB request context, 32 KiB target data, 128
7474
request headers totaling 64 KiB, 64 header mutations, 32 findings per stage,
7575
and 64 metadata entries. The external contract lives in
7676
`proto/supervisor_middleware.proto`, with service-author guidance in the
77-
[middleware operations guide](../docs/extensibility/supervisor-middleware/operations.mdx).
77+
[supported middleware operations](../docs/extensibility/supervisor-middleware/operations.mdx).
7878

7979
The work semaphore bounds aggregate buffered middleware input to approximately
8080
`32 × 4 MiB`, plus bounded envelope and parser overhead. It is a concurrency

‎docs/extensibility/supervisor-middleware/configure.mdx‎

Lines changed: 58 additions & 57 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,68 @@
11
---
22
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
33
# SPDX-License-Identifier: Apache-2.0
4-
title: "Configure and Operate Supervisor Middleware"
5-
sidebar-title: "Configure and Operate"
4+
title: "Supervisor Middleware Configuration"
5+
sidebar-title: "Configuration"
66
slug: "extensibility/supervisor-middleware/configure"
77
description: "Attach supervisor middleware to sandbox traffic, register your own middleware services, and observe middleware decisions."
88
keywords: "Supervisor Middleware, Configuration, Network Policy, Operations, OCSF, Logging"
99
---
1010

11-
You attach middleware to destination hosts in sandbox policy. Built-in middleware needs only a policy entry. Your own middleware service also needs a registration in gateway configuration.
11+
Register your middleware service in gateway configuration, then attach it to destination hosts in sandbox policy. This page also covers ordering, failure behavior, service lifecycle, and observability. Built-in middleware needs only a policy entry.
12+
13+
## Use Your Own Middleware Service
14+
15+
A middleware service is a gRPC server that implements the supervisor middleware protocol. The [content guard example](https://github.com/NVIDIA/OpenShell/tree/main/examples/supervisor-middleware-content-guard) implements request, response, and WebSocket checks in one service, and includes a policy and a local launcher.
16+
17+
To use your own service:
18+
19+
1. Start the service where both the gateway and sandbox supervisors can reach it.
20+
2. Register it in gateway configuration.
21+
3. Restart the gateway.
22+
4. Attach it in sandbox policy by setting `middleware` to the registration `name`.
23+
24+
Register the service in gateway TOML:
25+
26+
```toml
27+
[[openshell.supervisor.middleware]]
28+
name = "content-guard"
29+
grpc_endpoint = "https://content-guard.example:50051"
30+
tls_ca_cert_path = "/path/to/content-guard-ca.pem"
31+
max_payload_bytes = 262144
32+
timeout = "500ms"
33+
```
34+
35+
<ParamField path="name" type="string" required={true}>
36+
Unique name that policies use to attach the service. The `openshell/` prefix is reserved for built-ins.
37+
</ParamField>
38+
39+
<ParamField path="grpc_endpoint" type="string" required={true}>
40+
Service address reachable from the gateway and sandbox supervisors.
41+
</ParamField>
42+
43+
<ParamField path="max_payload_bytes" type="integer" required={true}>
44+
Largest request body, response body, or WebSocket message the service inspects. Must be at most 4 MiB and no larger than the service advertises.
45+
</ParamField>
46+
47+
<ParamField path="timeout" type="string" default="500ms">
48+
Time limit for `Describe`, `ValidateConfig`, and individual evaluations or stream exchanges. Use an integer followed by `ms` or `s`, from `10ms` through `30s`. A binding may advertise a shorter evaluation timeout. Accepted HTTP response and WebSocket streams have no connection-wide deadline.
49+
</ParamField>
50+
51+
<ParamField path="tls_ca_cert_path" type="string">
52+
Private CA for the service certificate. Without it, OpenShell uses platform trust roots.
53+
</ParamField>
54+
55+
<ParamField path="audience" type="string" default="urn:openshell:extension:middleware:<name>">
56+
Token audience. Refer to [Extension Authentication](/extensibility/extension-authentication).
57+
</ParamField>
58+
59+
<ParamField path="allow_insecure_transport" type="boolean" default="false">
60+
Allows a plaintext `http://` endpoint with no authentication. Use it for local development or on a network that already authenticates callers.
61+
</ParamField>
62+
63+
At startup, the gateway contacts every registered service to read its capabilities and verify [protocol compatibility](/extensibility/extension-negotiation). The gateway does not start if a service is unavailable or incompatible. [Gateway Configuration](/reference/gateway-config#supervisor-middleware-services) describes the full TOML context.
64+
65+
When the gateway has JWT signing configured, OpenShell authenticates every call to your service with a short-lived token. [Extension Authentication](/extensibility/extension-authentication) describes how your service validates it.
1266

1367
## Attach Middleware in Policy
1468

@@ -43,6 +97,7 @@ network_middlewares:
4397
order: 10
4498
endpoints:
4599
include: ["*.example.com"]
100+
exclude: ["trusted.example.com"]
46101
content-guard:
47102
name: Content guard
48103
middleware: content-guard
@@ -108,60 +163,6 @@ The map key, such as `content-guard`, is the entry's stable identity in logs. Ea
108163

109164
[Policy Schema](/reference/policy-schema#network-middleware) lists every field and limit.
110165

111-
## Use Your Own Middleware Service
112-
113-
A middleware service is a gRPC server that implements the supervisor middleware protocol. The [content guard example](https://github.com/NVIDIA/OpenShell/tree/main/examples/supervisor-middleware-content-guard) implements request, response, and WebSocket checks in one service, and includes a policy and a local launcher.
114-
115-
To use your own service:
116-
117-
1. Start the service where both the gateway and sandbox supervisors can reach it.
118-
2. Register it in gateway configuration.
119-
3. Restart the gateway.
120-
4. Attach it in sandbox policy by setting `middleware` to the registration `name`.
121-
122-
Register the service in gateway TOML:
123-
124-
```toml
125-
[[openshell.supervisor.middleware]]
126-
name = "content-guard"
127-
grpc_endpoint = "https://content-guard.example:50051"
128-
tls_ca_cert_path = "/path/to/content-guard-ca.pem"
129-
max_payload_bytes = 262144
130-
timeout = "500ms"
131-
```
132-
133-
<ParamField path="name" type="string" required={true}>
134-
Unique name that policies use to attach the service. The `openshell/` prefix is reserved for built-ins.
135-
</ParamField>
136-
137-
<ParamField path="grpc_endpoint" type="string" required={true}>
138-
Service address reachable from the gateway and sandbox supervisors.
139-
</ParamField>
140-
141-
<ParamField path="max_payload_bytes" type="integer" required={true}>
142-
Largest request body, response body, or WebSocket message the service inspects. Must be at most 4 MiB and no larger than the service advertises.
143-
</ParamField>
144-
145-
<ParamField path="timeout" type="string" default="500ms">
146-
Time limit for each call to the service. Must be 10 ms-30 s.
147-
</ParamField>
148-
149-
<ParamField path="tls_ca_cert_path" type="string">
150-
Private CA for the service certificate. Without it, OpenShell uses platform trust roots.
151-
</ParamField>
152-
153-
<ParamField path="audience" type="string" default="urn:openshell:extension:middleware:<name>">
154-
Token audience. Refer to [Extension Authentication](/extensibility/extension-authentication).
155-
</ParamField>
156-
157-
<ParamField path="allow_insecure_transport" type="boolean" default="false">
158-
Allows a plaintext `http://` endpoint with no authentication. Use it for local development or on a network that already authenticates callers.
159-
</ParamField>
160-
161-
At startup, the gateway contacts every registered service to read its capabilities and verify [protocol compatibility](/extensibility/extension-negotiation). The gateway does not start if a service is unavailable or incompatible. [Gateway Configuration](/reference/gateway-config#supervisor-middleware-services) describes the full TOML context.
162-
163-
When the gateway has JWT signing configured, OpenShell authenticates every call to your service with a short-lived token. [Extension Authentication](/extensibility/extension-authentication) describes how your service validates it.
164-
165166
## Choose Failure Behavior
166167

167168
Middleware fails when it cannot complete its check, for example when its service is unavailable, times out, returns an invalid result, or receives a payload over its limit. By default, OpenShell blocks the affected request, response, or WebSocket connection. The `on_error` field on a policy entry controls this behavior.

‎docs/extensibility/supervisor-middleware/index.mdx‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ OpenShell checks network policy first, so middleware only sees traffic that poli
4747

4848
Each middleware decides whether to allow, deny, or change the traffic. A denial blocks the request even when network policy allows it. If middleware fails or its service is unavailable, OpenShell blocks the affected traffic by default.
4949

50-
Middleware can handle three operations: HTTP requests, HTTP responses, and the text messages an agent sends over a WebSocket connection. Each middleware declares which operations it supports. [Middleware Operations](/extensibility/supervisor-middleware/operations) describes each one.
50+
Middleware can handle three operations: HTTP requests, HTTP responses, and the text messages an agent sends over a WebSocket connection. Each middleware declares which operations it supports. [Supported Operations](/extensibility/supervisor-middleware/operations) describes each one.
5151

5252
## Current Limitations
5353

@@ -63,7 +63,7 @@ To add middleware to your sandboxes, start with configuration:
6363

6464
<Cards>
6565

66-
<Card title="Configure and Operate" href="/extensibility/supervisor-middleware/configure">
66+
<Card title="Configuration" href="/extensibility/supervisor-middleware/configure">
6767

6868
Attach built-in middleware, register your own service, choose failure behavior, and observe middleware decisions.
6969
</Card>
@@ -85,15 +85,15 @@ Every middleware service implements these RPCs:
8585
- `Describe` returns the service manifest. The manifest lists the operations the service supports, with a payload limit and optional timeout for each. It also carries [protocol negotiation](/extensibility/extension-negotiation) metadata and the [expected audience](/extensibility/extension-authentication#confirm-the-audience-at-startup).
8686
- `ValidateConfig` checks the `config` object from a policy entry. The gateway calls it before it accepts a policy.
8787

88-
The service then implements an evaluation RPC for each operation it supports, as described in [Middleware Operations](/extensibility/supervisor-middleware/operations).
88+
The service then implements an evaluation RPC for each operation it supports, as described in [Supported Operations](/extensibility/supervisor-middleware/operations).
8989

9090
Most gRPC servers reject incoming messages larger than 4 MiB by default. Raise your server's limit to at least 4.5 MiB so that a maximum-size payload and the rest of the message fit.
9191

9292
These guides cover the rest of the service contract:
9393

9494
<Cards>
9595

96-
<Card title="Middleware Operations" href="/extensibility/supervisor-middleware/operations">
96+
<Card title="Supported Operations" href="/extensibility/supervisor-middleware/operations">
9797

9898
When OpenShell calls your service, what it receives, and what it can return for HTTP requests, HTTP responses, and WebSocket messages.
9999
</Card>

‎docs/extensibility/supervisor-middleware/operations.mdx‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,16 @@
11
---
22
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
33
# SPDX-License-Identifier: Apache-2.0
4-
title: "Middleware Operations"
5-
sidebar-title: "Operations"
4+
title: "Supported Operations"
5+
sidebar-title: "Supported Operations"
66
slug: "extensibility/supervisor-middleware/operations"
77
description: "Build a middleware service that inspects and changes sandbox HTTP requests, HTTP responses, and WebSocket messages."
88
keywords: "Supervisor Middleware, Operations, Bindings, HTTP, WebSocket, Request Body, Response Body, Header Mutation, Payload Limits"
99
---
1010

1111
A middleware service handles one or more operations, such as HTTP requests or WebSocket messages. This page describes when OpenShell calls your service for each operation, what your service receives, and what it can return. The RPCs that every middleware service implements are described in [Build Your Middleware Service](/extensibility/supervisor-middleware#build-your-middleware-service).
1212

13-
## Supported Operations
13+
## Operations and Phases
1414

1515
| Operation | Phase | When OpenShell calls your service | RPC |
1616
| --- | --- | --- | --- |
@@ -26,7 +26,7 @@ When several middleware match the same traffic, OpenShell calls them in ascendin
2626

2727
Every operation includes a request context. The context always includes `sandbox_id`. Use it for authorization, persistence, and correlation. The context also includes `sandbox` and `workspace` names when available. Names can be reused, so use them only for display and logging.
2828

29-
Operations that include headers deliver them in wire order, and repeated headers stay separate. OpenShell removes credential, routing, framing, and hop-by-hop headers before sending them to your service.
29+
Operations that include headers deliver them in wire order, and repeated headers stay separate. OpenShell removes credential, routing, and hop-by-hop headers before sending them to your service. It also removes framing headers from request input. Response preflight retains upstream `Content-Length`, `Content-Encoding`, and `Content-Range` as read-only metadata, unless `Connection` names them as hop-by-hop headers. Your service cannot change or remove these fields; OpenShell handles downstream framing.
3030

3131
Every result can include an optional reason code, findings, and metadata:
3232

@@ -109,7 +109,7 @@ OpenShell applies each middleware's changes as a unit. If one change is invalid,
109109

110110
### Responses
111111

112-
OpenShell calls your service after the external service responds and before the sandbox receives the response. Your service first decides whether to inspect the response, then can change or block it.
112+
OpenShell calls your service after the external service responds and before the sandbox receives the response. At preflight, your service can skip inspection, block delivery, or choose how to inspect the response. During inspection, it can change or block the response.
113113

114114
```mermaid
115115
sequenceDiagram
@@ -120,7 +120,7 @@ sequenceDiagram
120120
121121
Service-->>Supervisor: HTTP response
122122
Supervisor->>Middleware: Preflight with status and headers
123-
Middleware-->>Supervisor: Skip or inspect
123+
Middleware-->>Supervisor: Skip, inspect, or block
124124
Supervisor->>Middleware: Body, when inspecting
125125
Middleware-->>Supervisor: Pass, change, or stop
126126
Supervisor-->>Sandbox: Deliver response
@@ -129,7 +129,7 @@ sequenceDiagram
129129
<AccordionGroup>
130130
<Accordion title="Preflight and Body Modes">
131131

132-
The preflight includes the status and headers. A preflight result can skip the response, or inspect it with one body mode:
132+
The preflight includes the status and headers. A preflight result can skip inspection, block delivery with `block_delivery`, or inspect the response with one body mode:
133133

134134
| Mode | Behavior |
135135
| --- | --- |
@@ -239,7 +239,7 @@ A denied message closes the connection with code `1008`. When middleware fails,
239239
| Limit | Applies to | Value |
240240
| --- | --- | --- |
241241
| Payload per middleware | All operations | The registration's `max_payload_bytes`, up to 4 MiB. Applies to a request body, a whole response body or one streamed piece, or one text message, and to each replacement. |
242-
| Time per call | All operations | The smaller of the registration `timeout` and the timeout in the service manifest. |
242+
| Time per evaluation or stream exchange | All operations | The smaller of the registration `timeout` and the binding timeout in the service manifest. Accepted HTTP response and WebSocket streams have no connection-wide deadline. |
243243
| Streamed body piece | HTTP responses | 64 KiB. |
244244
| Time for all middleware on one streamed piece | HTTP responses | 30 seconds. |
245245
| Fragments per message | WebSocket messages | 4,096. |

‎docs/index.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -81,9 +81,9 @@ navigation:
8181
slug: supervisor-middleware
8282
path: extensibility/supervisor-middleware/index.mdx
8383
contents:
84-
- page: "Configure and Operate"
84+
- page: "Configuration"
8585
path: extensibility/supervisor-middleware/configure.mdx
86-
- page: "Operations"
86+
- page: "Supported Operations"
8787
path: extensibility/supervisor-middleware/operations.mdx
8888
- page: "Gateway Interceptors"
8989
path: extensibility/gateway-interceptors.mdx

‎docs/reference/gateway-config.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -370,7 +370,7 @@ The service `grpc_endpoint` supports plaintext `http://` and TLS `https://`. HTT
370370

371371
When `gateway_jwt` is configured, OpenShell attaches short-lived bearer credentials to gateway and supervisor calls and requires `https://`. A middleware endpoint must be reachable from sandbox supervisors, so Unix sockets are not an option here. Set `allow_insecure_transport = true` on a registration to keep a plaintext `http://` endpoint: OpenShell then attaches no credential, supervisors do not request one, and the gateway logs a warning naming the registration at every startup. mTLS client authentication, health checks, and runtime registration are not currently supported. The endpoint must be reachable from both the gateway and sandbox supervisors; use `host.openshell.internal` or another shared address that can be resolved in both places.
372372

373-
See [Configure and Operate Supervisor Middleware](/extensibility/supervisor-middleware/configure) for attachment, failure, and operational guidance.
373+
See [Supervisor Middleware Configuration](/extensibility/supervisor-middleware/configure) for attachment, failure, and operational guidance.
374374

375375
## Gateway Interceptors
376376

‎docs/reference/policy-schema.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -633,7 +633,7 @@ network_middlewares:
633633

634634
Host selectors use the same case-insensitive exact and DNS glob semantics as network endpoints: `*` matches exactly one DNS label and `**` matches one or more labels, so `**.example.com` covers subdomains but not `example.com` itself. Brace alternates are rejected at validation. A matching attachment joins only the operation chains its implementation advertises. An HTTP-only attachment may inspect a WebSocket upgrade GET without joining the post-upgrade chain; OpenShell permits the messages and records `binding_not_selected` coverage regardless of `on_error`. WebSocket bindings inspect complete client text messages. Binary messages pass with `unsupported_message_type` coverage for active stages. A fail-closed selector that can cover a `tls: skip` endpoint is rejected because OpenShell cannot inspect that traffic through any operation. An all-`fail_open` match may cover the endpoint; the supervisor bypasses the middleware and emits a detection finding.
635635

636-
See [Configure and Operate Supervisor Middleware](/extensibility/supervisor-middleware/configure) for registration, failure behavior, and operational guidance.
636+
See [Supervisor Middleware Configuration](/extensibility/supervisor-middleware/configure) for registration, failure behavior, and operational guidance.
637637

638638
## Full Example
639639

‎docs/sandboxes/policies.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -120,7 +120,7 @@ Matching entries run once each by ascending `order`; lower values run first, and
120120

121121
`on_error` defaults to `fail_closed`. Use `fail_open` only when skipping a selected stage that fails is acceptable. For WebSocket streams, a broken fail-open stage is disabled for the rest of that connection and OpenShell emits a state-change finding. A host-matched attachment joins only operation chains advertised by its implementation. An HTTP-only attachment may inspect the WebSocket upgrade GET, but post-upgrade messages pass with informational `binding_not_selected` coverage under either error mode. Binary messages also pass without middleware inspection and produce `unsupported_message_type` coverage for active WebSocket stages. Upstream-to-client messages remain uninspected. Policy validation rejects a fail-closed selector that can cover a `tls: skip` endpoint. An all-`fail_open` match may cover the endpoint; the supervisor bypasses the middleware and emits a detection finding.
122122

123-
See [Supervisor Middleware](/extensibility/supervisor-middleware) for an introduction, or [Configure and Operate Supervisor Middleware](/extensibility/supervisor-middleware/configure) for registration, ordering, failure behavior, and operations.
123+
See [Supervisor Middleware](/extensibility/supervisor-middleware) for an introduction, or [Supervisor Middleware Configuration](/extensibility/supervisor-middleware/configure) for registration, ordering, failure behavior, and operations.
124124

125125
## Baseline Filesystem Paths
126126

0 commit comments

Comments
 (0)