fix(deploy): blueprint MANAGER pattern everywhere — never hand-run the instance binary

CLAUDE.md

@@ -0,0 +1,57 @@
+# Inference Bazaar — agent rules
+
+## ⛔ Deploy the Blueprint Manager, NOT the operator binary
+
+This is the #1 recurring mistake. **Do not** run or `ExecStart` `inference-bazaar-operator run`
+directly, with a hardcoded `SERVICE_ID` / `TEST_MODE=true`. That bypasses the entire
+on-chain lifecycle and is a dev shortcut only.
+
+A production operator box runs the **Blueprint Manager daemon**:
+
+```
+ExecStart=/root/.cargo/bin/cargo-tangle blueprint run -t --pretty \
+  --http-rpc-url <HTTP_RPC> --ws-rpc-url <WS_RPC> \
+  --keystore-uri <KEYSTORE> --data-dir <DATA>/bpm-data \
+  --chain testnet --protocol tangle
+# systemd: SyslogIdentifier=blueprint-manager
+```
+
+The manager watches the chain and **spawns the per-service `inference-bazaar-operator run`
+instance itself** when a service request is approved. `inference-bazaar-operator run`
+(`operator/src/bin/blueprint.rs`) is the INSTANCE the manager spawns — it reads `SERVICE_ID`
+from the env the manager hands it; it is never launched by hand.
+
+**The lifecycle is two-sided and on-chain — honor every step:**
+
+1. Deploy the Blueprint Manager (the daemon above) on each operator box.
+2. The operator **registers** for the blueprint on-chain (`cargo tangle blueprint register`).
+3. A **user requests a service** selecting registered operators (`request-service`).
+4. Operators **approve**.
+5. The already-running Blueprint Manager **spawns the instance** with the assigned service id.
+
+Reference that does this correctly: `~/code/ai-trading-blueprint/deploy/go-live.sh` +
+`trading-blueprint.service`. When you write any deploy unit / script / doc, the `ExecStart`
+MUST be `cargo-tangle blueprint run`, and the runbook MUST include register → request → approve.
+
+`InferenceBazaarBSM.sol` (`[blueprint.manager]`) is the ON-CHAIN Blueprint Service Manager
+(lifecycle hooks/slashing) — a different thing from the off-chain manager daemon. Don't conflate.
+
+## Build / toolchain
+
+- Rust is pinned to **1.91** (`rust-toolchain.toml`). The `--features blueprint` build needs
+  **`protoc`** (a transitive dep): `PROTOC=/usr/bin/protoc cargo build … --features blueprint`.
+- CI gates: `cargo fmt --all --check`, core-crate `clippy -- -D warnings`
+  (`orderbook/settlement-core/matcher/settlement`), `cargo test --workspace`, `forge test`.
+
+## Operator inference backend (the "resell Tangle models" flow)
+
+To resell the Tangle Router's models, set `INFERENCE_BAZAAR_INFERENCE_URL=https://router.tangle.tools/v1`
++ `INFERENCE_BAZAAR_INFERENCE_API_KEY=<key>` → backend `mode="external"`, which passes the
+bonded-issuer check. The naked `INFERENCE_BAZAAR_ROUTER_URL` fallback is `mode="router"` and is
+**refused** for a bonded issuer (`operator/src/venue.rs`).
+
+## Other gotchas
+
+- **Local git branch refs go stale.** Always `git fetch` and check `origin/<default>` before
+  claiming something is unmerged — this has produced wrong "not merged" conclusions repeatedly.
+- The fleet is on Base **Sepolia**, not mainnet. The "USDC rail" RPC still points at testnet.

deploy/gen-resell-operator.sh

@@ -0,0 +1,119 @@
+#!/usr/bin/env bash
+# Generate a full-blueprint "resell" operator for the Hetzner fleet: the
+# Blueprint Manager systemd unit + the operator's settings.env. The operator
+# resells the Tangle Router's centralized models at a discount.
+#
+# ⛔ PRODUCTION = THE BLUEPRINT MANAGER, NOT THE INSTANCE BINARY.
+# This does NOT ExecStart `inference-bazaar-operator run` — that (with a hardcoded
+# SERVICE_ID / TEST_MODE) is for LOCAL TESTING ONLY. The unit runs the Blueprint
+# Manager daemon (`cargo-tangle blueprint run`); the manager watches the chain and
+# SPAWNS the per-service instance itself when a user's service request is approved.
+# See /home/drew/code/bazaar/CLAUDE.md and ai-trading-blueprint/deploy/go-live.sh.
+#
+# WHY the resell backend is valid (not the forbidden mode): the settings.env points
+# INFERENCE_BAZAAR_INFERENCE_URL at the router with the operator's own key → backend
+# mode="external", which passes the bonded-issuer check (operator/src/venue.rs). The
+# naked ROUTER_URL fallback would be mode="router" and is refused for an issuer.
+#
+# What you must provide (NOT in the repo — human-gated):
+#   OP_N            unique index for paths/unit name (e.g. 5; 3/4 already live)
+#   OPERATOR_KEY    EVM attester/signing key, 0x-hex (settlement attester; the
+#                   Tangle operator identity is imported into the keystore separately)
+#   SUBMITTER_KEY   gas/submitter key, 0x-hex (keep distinct from OPERATOR_KEY)
+#   ROUTER_API_KEY  this operator's OWN Tangle Router API key (the resell credential)
+#   INSTRUMENT      market to make, e.g. anthropic/claude-opus-4-8:output
+#
+# Usage:
+#   OP_N=5 OPERATOR_KEY=0x.. SUBMITTER_KEY=0x.. ROUTER_API_KEY=tngl-.. \
+#   INSTRUMENT=anthropic/claude-opus-4-8:output deploy/gen-resell-operator.sh
+set -euo pipefail
+cd "$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+
+: "${OP_N:?set OP_N -- unique operator index, e.g. 5}"
+: "${OPERATOR_KEY:?set OPERATOR_KEY -- 0x-hex EVM attester key}"
+: "${ROUTER_API_KEY:?set ROUTER_API_KEY -- this operator Tangle Router API key}"
+SUBMITTER_KEY="${SUBMITTER_KEY:-$OPERATOR_KEY}"
+INSTRUMENT="${INSTRUMENT:-anthropic/claude-opus-4-8:output}"
+ROUTER_URL="${ROUTER_URL:-https://router.tangle.tools}"
+SETTLEMENT_ADDR="${INFERENCE_BAZAAR_SETTLEMENT_ADDR:-0x64867eacf2e4581d182c2Be634cfD7fF3D3d9f83}"
+BLUEPRINT_ID="${BLUEPRINT_ID:-17}"
+HTTP_RPC="${HTTP_RPC_URL:-https://base-sepolia-rpc.publicnode.com/}"
+WS_RPC="${WS_RPC_URL:-wss://base-sepolia.drpc.org/}"
+ROOT="/opt/inference-bazaar${OP_N}"
+SETTINGS="deploy/hetzner/inference-bazaar${OP_N}.settings.env"
+UNIT="deploy/hetzner/inference-bazaar${OP_N}-blueprint-manager.service"
+
+# ── the operator's settings.env — the env the manager hands the spawned instance.
+#    NO SERVICE_ID here: the manager assigns it when the service request is approved.
+cat > "$SETTINGS" <<EOF
+# Inference Bazaar operator ${OP_N} — resell config. The Blueprint Manager passes
+# this to the instance it spawns on an approved service request.
+INFERENCE_BAZAAR_CHAIN_ID=${INFERENCE_BAZAAR_CHAIN_ID:-84532}
+INFERENCE_BAZAAR_SETTLEMENT_ADDR=${SETTLEMENT_ADDR}
+INFERENCE_BAZAAR_RPC_URL=${INFERENCE_BAZAAR_RPC_URL:-https://sepolia.base.org}
+INFERENCE_BAZAAR_OPERATOR_KEY=${OPERATOR_KEY}
+INFERENCE_BAZAAR_SUBMITTER_KEY=${SUBMITTER_KEY}
+INFERENCE_BAZAAR_SIDECAR_URL=http://127.0.0.1:9310
+INFERENCE_BAZAAR_ROUTER_URL=${ROUTER_URL}
+# RESELL backend: serve inference via the Tangle Router with this operator's key.
+# mode="external" (passes the bonded-issuer check) — NOT the forbidden router fallback.
+INFERENCE_BAZAAR_INFERENCE_URL=${ROUTER_URL}/v1
+INFERENCE_BAZAAR_INFERENCE_API_KEY=${ROUTER_API_KEY}
+INFERENCE_BAZAAR_INSTRUMENT=${INSTRUMENT}
+INFERENCE_BAZAAR_MM_SIZE=1000000
+INFERENCE_BAZAAR_MM_LEVELS=3
+EOF
+
+# ── the Blueprint Manager systemd unit (the daemon — it spawns the instance).
+cat > "$UNIT" <<EOF
+[Unit]
+Description=Inference Bazaar Blueprint Manager — operator ${OP_N} (blueprint ${BLUEPRINT_ID})
+After=network-online.target inference-bazaar-mm-sidecar.service
+Wants=network-online.target inference-bazaar-mm-sidecar.service
+
+[Service]
+Type=simple
+User=root
+WorkingDirectory=${ROOT}
+Environment=PATH=/root/.cargo/bin:/usr/local/bin:/usr/bin:/bin
+Environment=RUST_LOG=info,blueprint_manager=debug,inference_bazaar_operator=debug
+EnvironmentFile=${ROOT}/settings.env
+# The MANAGER. It watches the chain for approved service requests for blueprint
+# ${BLUEPRINT_ID} and spawns the inference-bazaar-operator instance itself.
+ExecStart=/root/.cargo/bin/cargo-tangle blueprint run --protocol tangle \\
+  --http-rpc-url ${HTTP_RPC} --ws-rpc-url ${WS_RPC} \\
+  --keystore-uri ${ROOT}/keystore --data-dir ${ROOT}/bpm-data \\
+  --chain testnet --settings-file ${ROOT}/settings.env
+Restart=always
+RestartSec=10
+SyslogIdentifier=blueprint-manager
+
+[Install]
+WantedBy=multi-user.target
+EOF
+
+OP_ADDR=$(command -v cast >/dev/null && cast wallet address --private-key "$OPERATOR_KEY" 2>/dev/null || echo "<cast wallet address --private-key \$OPERATOR_KEY>")
+
+cat <<RUNBOOK
+
+==> wrote ${UNIT}
+==> wrote ${SETTINGS}   (operator ${OP_N}, reselling ${ROUTER_URL} as ${INSTRUMENT})
+    operator address: ${OP_ADDR}
+
+Runbook (⚠ = human-gated). The manager spawns the instance — you never run it by hand:
+
+1. ⚠ Fund ${OP_ADDR} (gas + collateral token) and post collateral:
+     COLLATERAL_AMOUNT=20000000 OPERATOR_KEY=${OPERATOR_KEY} deploy/onboard-operator.sh
+2. Install on 178.104.232.124 (the manager + its keystore/settings):
+     ssh root@178.104.232.124 'mkdir -p ${ROOT}/keystore ${ROOT}/bpm-data'
+     scp ${SETTINGS} root@178.104.232.124:${ROOT}/settings.env
+     scp ${UNIT} root@178.104.232.124:/etc/systemd/system/
+3. ⚠ Import the operator's Tangle identity into the keystore + register for the blueprint:
+     cargo tangle blueprint register --blueprint-id ${BLUEPRINT_ID} --keystore-uri ${ROOT}/keystore
+4. Start the manager (it now waits for a service request):
+     ssh root@178.104.232.124 'systemctl daemon-reload && systemctl enable --now inference-bazaar${OP_N}-blueprint-manager'
+5. ⚠ A USER requests a service selecting these operators; operators APPROVE:
+     cargo tangle blueprint request-service --blueprint-id ${BLUEPRINT_ID}   # the buyer/demand side
+     # operators approve -> the running manager SPAWNS the instance with the assigned service id.
+
+RUNBOOK

deploy/hetzner/inference-bazaar-blueprint-runtime.service

@@ -1,5 +1,9 @@
+# ⛔ TEST/DEV ONLY — this runs the instance binary directly (TEST_MODE + hardcoded
+# SERVICE_ID), bypassing the on-chain lifecycle. PRODUCTION runs the Blueprint
+# Manager (cargo-tangle blueprint run), which spawns the instance on an approved
+# service request — see ../gen-resell-operator.sh and ../../CLAUDE.md.
 [Unit]
-Description=InferenceBazaar blueprint runtime (Base Sepolia blueprint 17 service 3)
+Description=InferenceBazaar blueprint runtime — TEST_MODE direct-run (Base Sepolia blueprint 17 service 3)
 After=network.target inference-bazaar-mm-sidecar.service inference-bazaar-arti.service
 Wants=inference-bazaar-mm-sidecar.service inference-bazaar-arti.service
 

deploy/hetzner/inference-bazaar3-blueprint-runtime.service

@@ -1,5 +1,8 @@
+# ⛔ TEST/DEV ONLY — runs the instance binary directly, bypassing the on-chain
+# lifecycle. PRODUCTION runs the Blueprint Manager (cargo-tangle blueprint run),
+# which spawns the instance on an approved service request. See ../gen-resell-operator.sh.
 [Unit]
-Description=InferenceBazaar venue 2 (Base Sepolia blueprint 17 service 3, operator 0x2420)
+Description=InferenceBazaar venue 2 — TEST_MODE direct-run (Base Sepolia blueprint 17 service 3, operator 0x2420)
 After=network.target inference-bazaar-mm-sidecar.service
 Wants=inference-bazaar-mm-sidecar.service
 

docs/OPERATOR_ONBOARDING.md

@@ -169,20 +169,39 @@ A ready-to-edit template lives at [`.env.operator.example`](../.env.operator.exa
 
 ## 7. Run
 
-**Standalone bonded issuer (HTTP venue + on-chain settlement):**
+### Local testing only — run the binary directly
+
+For kicking the tires on your own box, run the lite venue directly. It goes
+on-chain the moment `CHAIN_ID` + `SETTLEMENT_ADDR` are set:
 
 ```bash
 cargo run -p inference-bazaar-operator --bin inference-bazaar-operator-lite
-# (same lite binary; it goes on-chain the moment CHAIN_ID + SETTLEMENT_ADDR are set)
 ```
 
-**Full blueprint runner (registered on Tangle, multi-operator CLOB):**
+> ⛔ **Do NOT do this in production.** Running `inference-bazaar-operator run`
+> directly (with a hardcoded `SERVICE_ID` / `TEST_MODE=true`) bypasses the entire
+> on-chain lifecycle. It's a dev shortcut only.
+
+### Production — run the Blueprint Manager (it spawns the instance)
+
+Each production operator box runs the **Blueprint Manager daemon**. The manager
+watches the chain and **spawns the `inference-bazaar-operator run` instance itself**
+when a user's service request is approved — you never `ExecStart` the instance binary:
 
 ```bash
-cargo run -p inference-bazaar-operator --bin inference-bazaar-operator --features blueprint -- run
+cargo-tangle blueprint run --protocol tangle \
+  --http-rpc-url <HTTP_RPC> --ws-rpc-url <WS_RPC> \
+  --keystore-uri <KEYSTORE> --data-dir <DATA>/bpm-data \
+  --chain testnet --settings-file <DATA>/settings.env
+# systemd SyslogIdentifier=blueprint-manager
 ```
 
-Kick a quote cycle (or let the tick-keeper do it every ~5 min):
+Your operator config (the env in §6 / `.env.operator`) goes in `settings.env`,
+which the manager passes to the instance it spawns. The full register → request →
+approve → spawn flow is §8. `deploy/gen-resell-operator.sh` generates this unit +
+settings file for a resell operator. Reference: `ai-trading-blueprint/deploy/go-live.sh`.
+
+Once an instance is live, kick a quote cycle (or let the tick-keeper do it every ~5 min):
 
 ```bash
 curl -X POST http://127.0.0.1:9100/mm-tick \
@@ -206,15 +225,18 @@ local command — it's the Tangle service lifecycle:
 deploy/base-sepolia.sh                      # cargo tangle blueprint deploy tangle …
 
 # Then, as the operator:
-cargo tangle blueprint register   --blueprint-id <ID> …
+cargo tangle blueprint register   --blueprint-id <ID> --keystore-uri <KEYSTORE>
+# A USER requests a service selecting registered operators:
 cargo tangle blueprint request-service --blueprint-id <ID> …
-# An approver (governance/Safe) calls approve on-chain; once approved you appear
-# in the service's operator set and the runner above serves under SERVICE_ID.
+# An approver (governance/Safe) approves on-chain; once approved, the Blueprint
+# Manager you started in §7 SPAWNS your instance with the assigned SERVICE_ID.
+# You never run the instance binary yourself — the manager does.
 ```
 
-The production runtime unit (`deploy/hetzner/inference-bazaar-blueprint-runtime.service`)
-shows the full env a registered operator runs with (`BLUEPRINT_ID`,
-`SERVICE_ID`, `TANGLE_CONTRACT`, `STAKING_CONTRACT`, keystore, data dir).
+The production unit is the **Blueprint Manager** (§7; `deploy/gen-resell-operator.sh`
+generates one). The older `deploy/hetzner/inference-bazaar-blueprint-runtime.service`
+runs the instance binary directly with `TEST_MODE=true` + a hardcoded `SERVICE_ID`
+— that is a **test/dev** config, not the production pattern.
 
 ---