Skip to content

docs(verification): refresh Orca live smoke evidence for 1.4.177 - #10

Merged
Bloh09 merged 1 commit into
mainfrom
fm/fm-orca-verification-refresh
Aug 10, 2026
Merged

docs(verification): refresh Orca live smoke evidence for 1.4.177#10
Bloh09 merged 1 commit into
mainfrom
fm/fm-orca-verification-refresh

Conversation

@Bloh09

@Bloh09 Bloh09 commented Aug 10, 2026

Copy link
Copy Markdown
Owner

Intent

Refresh docs/verification/runtime-backends.md#orca live smoke evidence on current Orca (>=1.4.177).

Context: the recorded Orca smoke evidence predates the current app version, and the composite '::' worktree-id behavior changed - a bare uuid now errors with worktree_id_requires_full_path - which the fm-orca-id-validation fix (landed as commit a2ba0ef, PR #4) targets. The evidence must reflect current reality.

Requirements:

  1. Read docs/verification/runtime-backends.md#orca for the recorded smoke set and evidence format, and bin/backends/orca.sh for the current adapter surface.
  2. Drive the recorded smoke set from an ISOLATED scratch firstmate home (never the operator real home): spawn a disposable smoke task through bin/fm-spawn.sh --backend orca, verify metadata records backend/terminal/composite worktree id/worktree path, exercise fm-peek/fm-send/fm-crew-state, then fm-teardown. Clean all disposable smoke artifacts.
  3. Specifically re-verify the composite worktree-id contract both ways: composite '::' accepted end-to-end, bare uuid rejected with worktree_id_requires_full_path.
  4. Update the #orca section's dated evidence with the current Orca version and observed outputs; correct any stale claims.

Acceptance: #orca evidence re-dated on current Orca with the composite-id behavior verified both ways, zero smoke artifacts left behind, no-mistakes pipeline green with a PR on origin (Bloh09/firstmate) against main.

What Changed

  • Re-dated the #orca section of docs/verification/runtime-backends.md to a 2026-08-10 live smoke run against Orca app version 1.4.177, adding the observed result.runtime.appVersion=1.4.177 field and recording that orca worktree create returns a composite <repo-id>::<absolute-path> worktree id.
  • Added a "Composite worktree id contract" subsection documenting both sides of the id behavior: the full composite selector is accepted end-to-end, while a bare uuid is rejected with worktree_id_requires_full_path, along with the observed outputs from a live fm-spawn.sh --backend orca scout, fm-peek/fm-send/fm-crew-state exercises, and guarded teardown ending in selector_not_found.
  • Listed tests/fm-teardown-endpoint-safety.test.sh in the recorded smoke suite and noted the 2026-08-09 composer-shape captures remain current under 1.4.177.

Risk Assessment

✅ Low: Docs-only evidence refresh whose source-verifiable claims (validator semantics, test coverage, file references) all match the code, with no source changes and no leftover smoke artifacts in the repo.

Testing

Re-ran the doc's recorded Orca smoke set live on Orca 1.4.177 from an isolated scratch firstmate home (spawn, metadata, composite worktree-id accepted and bare uuid rejected with worktree_id_requires_full_path, peek/send/crew-state, guarded teardown with selector_not_found postcheck) plus the four cited fake-Orca test suites; every refreshed doc claim matched observed behavior and all smoke artifacts were cleaned up.

Evidence: fm-spawn scout success line (backend=orca)
spawned orca-doc-smoke-x7 harness=echo kind=scout window=fm-orca-doc-smoke-x7 worktree=/Users/brandon/orca/workspaces/firstmate/fm-orca-doc-smoke-x7
Evidence: Task metadata with composite orca_worktree_id
window=fm-orca-doc-smoke-x7
endpoint_task_id=orca-doc-smoke-x7
worktree=/Users/brandon/orca/workspaces/firstmate/fm-orca-doc-smoke-x7
project=/Users/brandon/firstmate/projects/firstmate
harness=echo
launch_profile=raw
kind=scout
tasktmp=/tmp/fm-orca-doc-smoke-x7
model=default
effort=default
backend=orca
orca_worktree_id=75c42b61-eee5-47da-bcd6-4f81d5ddba7f::/Users/brandon/orca/workspaces/firstmate/fm-orca-doc-smoke-x7
terminal=term_c9c93679-f461-4a3e-9e27-31fb1999be0f
Evidence: Composite-id contract verified both ways on Orca 1.4.177

id:<repo-id>::<path> -> ok=true id:<bare-uuid> -> ok=false error.code=worktree_id_requires_full_path message: "Worktree id selectors must use the full <repo-id>::<path> value. Use the id from orca worktree list --json, or target by path:<path>, branch:<branch>, or issue:<number>."

=== composite id accepted ===
{
    "id": "bad0c09d-7b05-4473-8cc5-1108565c28d0",
    "ok": true,
    "result": {
        "worktree": {
            "id": "75c42b61-eee5-47da-bcd6-4f81d5ddba7f::/Users/brandon/orca/workspaces/firstmate/fm-orca-doc-smoke-x7",
            "instanceId": "3a772eda-cf5b-4580-843a-53d22bfe3f94",
            "repoId": "75c42b61-eee5-47da-bcd6-4f81d5ddba7f",
            "projectId": "github:kunchenguid/firstmate",
            "hostId": "local",
            "projectHostSetupId": "75c42b61-eee5-47da-bcd6-4f81d5ddba7f",
            "path": "/Users/brandon/orca/workspaces/firstmate/fm-orca-doc-smoke-x7",
            "head": "71e26bd32476d604ca0d3b8b02984e21f4da6dbb",
            "branch": "refs/heads/Bloh09/fm-orca-doc-smoke-x7",
            "isBare": false,
            "isMainWorktree": false,
            "displayName": "fm-orca-doc-smoke-x7",
            "comment": "",
            "linkedIssue": null,
            "linkedPR": null,

=== bare uuid rejected ===
{
  "id": "96b237c7-add5-45a7-924f-000c11d8a51d",
  "ok": false,
  "error": {
    "code": "worktree_id_requires_full_path",
    "message": "Worktree id selectors must use the full <repo-id>::<path> value. Use the id from `orca worktree list --json`, or target by path:<path>, branch:<branch>, or issue:<number>."
  },
  "_meta": {
    "runtimeId": "8a14d302-da0b-49cd-83f2-b4e1ee468f4a"
  }
}
Evidence: fm-peek live shell capture
=== fm-peek ===
●━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
●  WATCHER DOWN - SUPERVISION IS OFF
●  1 task(s) in flight, but no watcher has a fresh beacon (last beat: never, grace 300s).
●  Trust the emitted supervision protocol for this harness; do not use shell & for watcher repair.
●  This is a supervision warning only; the guarded operation WILL still run.
●  watcher supervision needs Stop-owned automatic recovery; inspect the hook registration and startup status before ending the turn.
●━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━


fm-orca-doc-smoke-ready
brandon@Brandons-MacBook-Pro fm-orca-doc-smoke-x7 %
Evidence: fm-send verdict=unknown on raw shell
=== fm-send ===
WARNING: watcher still down (same stale episode; last beat: never, grace 300s) - full banner already printed this episode.
error: text not submitted to term_c9c93679-f461-4a3e-9e27-31fb1999be0f (delivery unconfirmed; verdict=unknown; tried meta=/tmp/fm-orca-doc-smoke-3364/home/state/orca-doc-smoke-x7.meta; backend=from-meta)
send exit=0
Evidence: Post-send peek and fm-crew-state structured line
=== fm-peek after send ===


fm-orca-doc-smoke-ready

fm-orca-doc-smoke-send-ok
brandon@Brandons-MacBook-Pro fm-orca-doc-smoke-x7 %

=== fm-crew-state ===
usage: fm-crew-state.sh <id>
=== fm-crew-state orca-doc-smoke-x7 ===
state: unknown · source: pane · harness state unavailable (unknown missing)
Evidence: Guarded teardown then selector_not_found and exited terminal
complete: orca-doc-smoke-x7 decision inventory reviewed
=== fm-teardown (after decision-hold complete) ===
WARNING: watcher still down (same stale episode; last beat: never, grace 300s) - full banner already printed this episode.
teardown: reaping leaked worktree process(es) for orca-doc-smoke-x7: 5775 5836
teardown: force-killing leaked worktree process(es) for orca-doc-smoke-x7: 5775 5836
teardown orca-doc-smoke-x7 complete (window term_c9c93679-f461-4a3e-9e27-31fb1999be0f, worktree /Users/brandon/orca/workspaces/firstmate/fm-orca-doc-smoke-x7)
Backlog: orca-doc-smoke-x7 just finished. Run tasks-axi done orca-doc-smoke-x7 --report data/orca-doc-smoke-x7/report.md, then run tasks-axi ready for dependency-cleared candidates, check date gates, and dispatch only work whose blockers are gone and date is due.

=== post-teardown worktree show ===
{
  "id": "f5dcb9ad-5864-4df9-9beb-80126df27e96",
  "ok": false,
  "error": {
    "code": "selector_not_found",
    "message": "selector_not_found"
  },
  "_meta": {
    "runtimeId": "8a14d302-da0b-49cd-83f2-b4e1ee468f4a"
  }
}

=== post-teardown terminal read ===
{
  "id": "8c165836-2dc7-4365-8089-79ed2804d387",
  "ok": true,
  "result": {
    "terminal": {
      "handle": "term_c9c93679-f461-4a3e-9e27-31fb1999be0f",
      "status": "exited",
      "tail": [],
      "truncated": false,
      "limited": false,
      "oldestCursor": "0",
      "nextCursor": "0",
      "latestCursor": "0",
      "returnedLineCount": 0
    }
  },
  "_meta": {
    "runtimeId": "8a14d302-da0b-49cd-83f2-b4e1ee468f4a"
  }
}
Evidence: Zero smoke artifact residue audit
=== orca worktree list (smoke residue?) ===
0
0 matches
=== path removed? ===
ls: /Users/brandon/orca/workspaces/firstmate/fm-orca-doc-smoke-x7: No such file or directory
=== git worktree/branch residue in project repo ===
no worktree entry
no branch
=== tasktmp ===
ls: /tmp/fm-orca-doc-smoke-x7: No such file or directory

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

✅ **Review** - passed

✅ No issues found.

✅ **Test** - passed

✅ No issues found.

  • tests/fm-backend-orca.test.sh (fake-Orca adapter suite, pass)
  • tests/fm-backend.test.sh (backend dispatcher suite, pass)
  • tests/fm-bootstrap.test.sh (bootstrap suite, pass)
  • tests/fm-teardown-endpoint-safety.test.sh (composite-id validation pins, pass)
  • Live orca status --json confirming reachable=true, state=ready, appVersion=1.4.177
  • Live bin/fm-spawn.sh orca-doc-smoke-x7 &lt;project&gt; --scout --backend orca from an isolated scratch FM_HOME with a raw-shell launch command
  • Inspected state/orca-doc-smoke-x7.meta for backend=orca, terminal=term_..., composite orca_worktree_id, matching worktree path
  • Live orca worktree show id:&lt;repo-id&gt;::&lt;path&gt; (ok=true) and id:&lt;bare-uuid&gt; (worktree_id_requires_full_path with doc-identical message)
  • Live bin/fm-peek.sh, bin/fm-send.sh (verdict=unknown on raw shell, line delivered and executed), bin/fm-crew-state.sh (structured state line)
  • Live bin/fm-teardown.sh: observed the decision-hold gate refusal, completed it via bin/fm-decision-hold.sh complete --none, then confirmed terminal closed and post-teardown orca worktree show returned selector_not_found
  • Cleanup audit: no Orca worktree/terminal residue, no git branch/worktree residue in the project repo, scratch home and tasktmp removed, gate worktree clean
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

Re-date the #orca maintainer verification record on current Orca
appVersion 1.4.177, including composite <repo-id>::<path> worktree-id
acceptance and bare-uuid worktree_id_requires_full_path refusal.
@Bloh09
Bloh09 merged commit 4c307eb into main Aug 10, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants