fix(orchestration): reliable agent-to-agent DM replies (surface, correct content, resilience) — Closes #4583#4599
Merged
senamakel merged 3 commits intoJul 6, 2026
Conversation
… dropped A wrapped Claude harness stamps `message.line = 0` on every DM, so after tinyhumansai#4582 keyed the session on the shared `wrapper_session_id` the wake idempotence cursor (`latest_seq > cursor`) pinned at 0 after the first message — every later DM was persisted + acked but silently skipped the graph (no reply). Replace the diagnostic-only WARN guard with a store-assigned, strictly-increasing per-`(agent, session)` ingest ordinal (`store::next_session_seq`), so every new DM advances `last_seq` past the cursor and wakes the graph. Self-migrating and also fixes the secondary `cycle_id` collision. Refs tinyhumansai#4583
…ry on failure Three outbound-path fixes on top of tinyhumansai#4582: - Reply content: the front end's channel response used `run_single`'s trailing narration ("Done — sent to the session") and discarded the `reply_to_channel` argument (the real answer). Capture the decision-tool payload via `with_decision_capture` and send that instead (same for `defer_to_orchestrator` → reasoning instructions). - Surface in UI: persist the agent's own outbound reply (`role = owner`) and emit `notify_orchestration_message`, so it appears in the orchestration chat window. Ingest only persisted inbound DMs and the UI live-refetches only on that socket event, so replies never showed. - Resilience: a transient send/relay/provider error (e.g. a staging `HTTP 400: body must be encrypted ciphertext` on `send_dm`) made the node throw → the whole wake failed, the cursor stayed put, and the one-shot wake left the message orphaned in silence. Retry the wake with backoff (5s/15s/45s); it resumes from the graph checkpoint so retries only re-attempt the failed tail. Refs tinyhumansai#4583
Contributor
📝 WalkthroughWalkthroughThis PR assigns monotonic ingest sequences inside the store, uses that sequence for session/message persistence, adds wake retry/backoff, persists outbound replies after successful sends, and captures decision tool payloads so frontend agent nodes can return tool-authored text. ChangesWake Cursor Sequencing Fix
Estimated code review effort: 4 (Complex) | ~40 minutes Wake Retry, Reply Persistence, and Decision Capture
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Harness
participant persist_message
participant Store
Harness->>persist_message: classified message (line may repeat/reset)
persist_message->>Store: next_session_seq(agent_id, session_id)
Store-->>persist_message: ingest_seq (MAX(seq)+1)
persist_message->>Store: set OrchestrationMessage.seq = ingest_seq
persist_message->>Store: set OrchestrationSession.last_seq = ingest_seq
sequenceDiagram
participant send_dm
participant Signal
participant persist_outgoing_reply
participant Store
participant OrchestrationBus
send_dm->>Signal: send message
Signal-->>send_dm: success
send_dm->>persist_outgoing_reply: reply text
persist_outgoing_reply->>Store: write OrchestrationMessage
persist_outgoing_reply->>OrchestrationBus: notify UI refetch
Possibly related PRs
Suggested labels: Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Comment |
coderabbitai
Bot
added
agent
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a06cedb438
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/openhuman/orchestration/store.rs`:
- Around line 223-229: The `next_session_seq` and `insert_message` flow is
vulnerable to duplicate `seq` values because `persist_message` and
`persist_outgoing_reply` compute `MAX(seq)+1` and then insert using separate
calls on fresh connections from `with_connection`. Update the persistence path
so the read and write happen in a single SQLite transaction, and make the
transaction cover the `next_session_seq`/`insert_message` pair used by both
`persist_message` and `persist_outgoing_reply` to ensure concurrent writers
cannot race on the same `(agent_id, session_id)`.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 859402ef-5fa1-4273-a69f-c19f71f30d28
📒 Files selected for processing (4)
src/openhuman/orchestration/ingest.rssrc/openhuman/orchestration/ops.rssrc/openhuman/orchestration/store.rssrc/openhuman/orchestration/tools.rs
…mp wake last_seq - Wrap `MAX(seq)+1` → `INSERT` in a `BEGIN IMMEDIATE` txn (`store::in_immediate_txn`) with a `busy_timeout`, so the drain's inbound persist and the graph's `send_dm` reply persist can't race on the same `(agent, session)` and write a duplicate `seq` (which would break the monotonic wake cursor). Adds a multi-thread concurrency test. [CodeRabbit] - `persist_outgoing_reply` no longer advances `sessions.last_seq` for our own outbound reply — the wake cursor only tracks inbound seqs, so bumping it made `ingest_cursor_lag` / `orchestration.status` falsely report pending work until the next inbound DM. `upsert_session`'s `MAX(..)` clamp keeps the inbound `last_seq`; passing 0 refreshes `last_message_at` only. [Codex] Refs tinyhumansai#4583
Contributor
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/openhuman/orchestration/store.rs (1)
249-262: 🚀 Performance & Scalability | 🟡 Minor | ⚡ Quick winAdd an index for
next_session_seq
next_session_seqfiltersmessagesby(agent_id, session_id)and then computesMAX(seq), but the only existing index is(agent_id, session_id, timestamp). Add an index that includesseq(for example,(agent_id, session_id, seq)) so this hot allocation stays cheap under write lock as sessions grow.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/openhuman/orchestration/store.rs` around lines 249 - 262, `next_session_seq` currently scans `messages` by `agent_id` and `session_id` to compute `MAX(seq)`, so add a supporting index that includes `seq` for this lookup pattern. Update the schema/migration near the `messages` table indexes to add an index on `agent_id, session_id, seq` so the `next_session_seq` query in `store.rs` stays efficient as session history grows.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@src/openhuman/orchestration/store.rs`:
- Around line 249-262: `next_session_seq` currently scans `messages` by
`agent_id` and `session_id` to compute `MAX(seq)`, so add a supporting index
that includes `seq` for this lookup pattern. Update the schema/migration near
the `messages` table indexes to add an index on `agent_id, session_id, seq` so
the `next_session_seq` query in `store.rs` stays efficient as session history
grows.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: d6fb7c32-f9bb-4ab3-a46e-f176e67564f9
📒 Files selected for processing (3)
src/openhuman/orchestration/ingest.rssrc/openhuman/orchestration/ops.rssrc/openhuman/orchestration/store.rs
🚧 Files skipped from review as they are similar to previous changes (2)
- src/openhuman/orchestration/ingest.rs
- src/openhuman/orchestration/ops.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #4582. Fixes four defects in the tiny.place agent-to-agent orchestration DM loop, all reproduced + verified end-to-end on staging:
message.line = 0on every DM, so under fix(orchestration): key inbound DMs on the shared per-pair session id #4582's sharedwrapper_session_idkey the wake cursor pinned at 0 and every DM after the first was persisted + acked but never woke the graph (no reply). Now stamps a store-assigned monotonic per-(agent, session)ingest ordinal.run_single's trailing narration ("Done — sent to the session") instead of thereply_to_channelargument. Now captures + sends the decision-tool payload.role=owner+ emitsnotify_orchestration_message.send_dmerror (e.g. stagingHTTP 400: body must be encrypted ciphertext) failed the whole wake and left the message in permanent silence. Now retries the wake with backoff (resumes from the graph checkpoint).Problem
#4582 keyed inbound sessions on the shared
wrapper_session_idbut the wake idempotence cursor still ridesseq = env.message.line, which is0for every message from a wrapped Claude harness (its own #4583 guard only warned). Beyond that root cause, the outbound path had three independent gaps: the reply text sent was the model's post-tool narration, the reply was never recorded in the store the UI renders from, and any transient send/relay error orphaned the message because the cursor only advances on success and the wake is one-shot.Solution
store::next_session_seq=MAX(seq)+1per(agent, session);persist_messagestamps it on both the sessionlast_seqand the messageseq, replacing the diagnostic WARN guard. Monotonic regardless of the wireline; also fixes the secondarycycle_idcollision.with_decision_capture(task-local) records thereply_to_channel/defer_to_orchestratorargument;frontend_instruct/frontend_compileprefer it over the raw turn text.persist_outgoing_replywrites the agent's reply (role=owner, monotonic seq, no wake trigger) and fans outnotify_orchestration_message, mirroring the inbound path and thesend_masterRPC.schedule_wakeretriesinvoke_orchestration_graphon error with 5s/15s/45s backoff, bailing if a newer wake supersedes it; the graph checkpoints per super-step so retries only re-attempt the failed tail (no repeated LLM work).Submission Checklist
persist_stamps_monotonic_ingest_seq_so_line_zero_dms_still_wake,decision_capture_surfaces_tool_payload_not_turn_narration,decision_capture_is_none_without_a_decision_tool; existing cursor/idempotence tests still pass (58/58 orchestration).next_session_seq, ingest stamping, decision capture) is unit-covered; the graph-runtime wiring (persist_outgoing_reply,notify, the retry loop) is behaviour-only and was verified live on staging. Will add targeted coverage ifdiff-coverflags it in CI.[orchestration]).Closes #4583in## Related.Impact
src/openhuman/orchestration/*) only; no Tauri/app/mobile/CLI changes. Gated behind[orchestration].enabled(Beta).Related
send_dmbeforecompress/world_diffto cut ~20s time-to-reply; scope the graph checkpoint thread id to(agent, session);SessionEnvelopeV1inReplyTo/fromSessionforsend_and_waitcorrelation (needs SDK/peer coordination).AI Authored PR Metadata (required for Codex/Linear PRs)
Linear Issue
Commit & Branch
fix/orchestration-outbound-reply-and-wake-resilienceb0c376df4,a06cedb43Validation Run
pnpm --filter openhuman-app format:check— N/A (no app/TS changes);cargo fmtapplied to changed files.pnpm typecheck(no TypeScript changed).cargo test --lib openhuman::orchestration::→ 58 passed, 0 failed.cargo fmtapplied; orchestration lib compiles clean.app/src-taurichanges).Validation Blocked
command:N/Aerror:N/Aimpact:N/ABehavior Changes
Summary by CodeRabbit
New Features
Bug Fixes