Skip to content

fix(runtime): gate provider reasoning replay by source model - #4286

Merged
Astro-Han merged 16 commits into
apache:mainfrom
Astro-Han:fix/reasoning-replay-provenance
Aug 31, 2026
Merged

fix(runtime): gate provider reasoning replay by source model#4286
Astro-Han merged 16 commits into
apache:mainfrom
Astro-Han:fix/reasoning-replay-provenance

Conversation

@Astro-Han

@Astro-Han Astro-Han commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Summary

Make provider-owned reasoning replay use one fail-closed provider-state contract across normal turns, continuations, compaction, overflow recovery, and durable reload.

  • Persist and replay Anthropic signed and redacted_thinking blocks through ModelAdapter → thinking RuntimeEvent → the existing AI SDK request converter.
  • Reuse the existing openai-chat-plaintext contract for GitHub Copilot openai-chat, preserving observed reasoning_content request-field behavior.
  • Prepare the provider target once per backend activation, then use that same resolved target both to persist one opaque providerStateIdentity on the Run-opening header and to build the backend. The identity covers immutable connection ID, provider type, effective endpoint, connection-credential version, and request-header credential version.
  • Replay provider-owned reasoning only when source providerStateIdentity + modelId exactly matches the current target. Missing legacy provenance fails closed while text, tool calls, and tool results remain.
  • Bind safe-boundary continuation admission to that same target identity and ordered replay projection; bump the durable provider replay projection to v2.
  • Keep persisted v1 claims readable and migration-safe, but explicitly reject them for v2 replay instead of interpreting their old digest under new semantics.

Root cause

The original contract treated connectionId + modelId as the exact provider route. That is not true in this repository: a connection keeps its ID when its endpoint, API key/OAuth material, or custom request-header credential is replaced. A same-ID next turn could therefore replay signed, redacted, or encrypted reasoning created by another relay or account.

A second versioning defect came from changing continuation admission semantics without changing PROVIDER_REPLAY_PROJECTION_VERSION. The new digest included target route state while persisted v1 claims had been produced without it. Rejection was safe, but the durable protocol version no longer described the bytes it authenticated.

Both findings have one owner-level correction: prepare the provider target once inside the backend-activation transition, persist its identity on the existing Run route-provenance record, and build from that same prepared target. This removes the temporary two-read authority split between Run admission and backend construction. RuntimeEvent remains the canonical transcript/execution ledger; event.runId joins to the Run-opening provenance needed to decide whether opaque provider state may cross the current boundary. Durable continuation planning and resume are separate time boundaries, so each deliberately prepares once and the v2 digest rejects any intervening provider-state change.

Lifecycle and replay exits

flowchart TD
  P["Host RuntimePolicy authority<br/>connection + endpoint + credential versions"] --> G["RuntimePolicyActivationGate"]
  G --> A["Prepare provider target once<br/>resolved target + opaque identity"]
  A --> R["Persist identity + model on Run-opening header"]
  A --> B["Build backend from the same prepared target"]
  B --> D["Provider dispatch"]
  D --> M["ModelAdapter<br/>ordered reasoning boundaries"]
  M --> E["RuntimeEvent durable ledger<br/>thinking, text, tools, results"]

  E --> J["Replay projection"]
  R --> J
  A --> T["Current target identity + model"]
  T --> J
  J --> C{"Source Run identity + model<br/>exactly match target?"}
  C -- yes --> H["Admit provider-owned reasoning"]
  C -- no or missing --> O["Omit provider-owned reasoning only"]
  J --> K["Always preserve portable text and tool evidence"]
  H --> S["AiSdkBackend materializer"]
  O --> S
  K --> S
  S --> Z["Existing AI SDK request converter"]

  CP["Continuation planning boundary<br/>prepare once and seal v2 digest"] --> CR["Durable continuation claim"]
  CR --> CE["Continuation resume boundary<br/>prepare once and recompute v2 digest"]
  CE --> V{"Admitted projection still exact?"}
  V -- no --> X["Reject before dispatch"]
  V -- yes --> S

  L["Persisted projection v1"] --> L2["Readable after schema migration"]
  L2 --> L3["Explicit unsupported-version rejection"]
Loading

The Host computes identity; Runtime owns replay admission; ModelAdapter remains the only AI SDK boundary; RuntimeKernel and ToolRuntime continue to own execution, permissions, concurrency, and recovery. No provider-specific serializer, RuntimeEvent provenance copy, providerOptions side channel, SDK prepareStep/stopWhen loop, or second provider-state authority was added.

The broader Run-header retirement/migration remains tracked in #4311 and #4283. This PR extends the current Run-opening provenance seam by one opaque identity because that is the smallest durable fact that can authenticate provider-owned replay today.

Compatibility and risk

  • Legacy Run headers without providerStateIdentity continue to decode; only provider-owned reasoning is omitted.
  • SQLite runtime schema v15 preserves existing v1 continuation rows and admits v2 rows. Readers accept v1/v2, while current replay requires v2 and rejects unknown versions.
  • Endpoint, primary credential, and request-header credential replacement all change the identity even when connection ID and model remain unchanged.
  • RuntimePolicy mutation and backend activation are serialized by the existing activation gate. Within one activation, identity persistence and backend construction consume the same prepared target; across durable continuation planning/resume boundaries, the target is re-observed and the v2 digest must still match.
  • Existing OpenAI Codex V3 checkpoint connection/model semantics remain separate and unchanged; this PR does not conflate that durable checkpoint contract with RuntimeEvent reasoning admission.

Commit structure

The original provider-contract and recovery series remains independently reviewable. The two review corrections added here are:

  1. feat(runtime): read provider replay projection v2
  2. fix(runtime): bind reasoning replay to provider state
  3. fix(runtime): prepare provider target once per activation
  4. test(storage): follow runtime schema authority
  5. test(runtime-host): compose continuation planning authority
  6. test(runtime): align replay assertions with node test

The first is compatibility-only: read v1/v2, migrate storage, reject unknown versions. The second switches current admission to v2 and introduces the Host-owned identity. The third removes the interim double-read shape by making one prepared target own both Run provenance and backend construction. The fourth removes a duplicated schema-version literal from the migration test. The fifth composes the planning authority in the pre-claim Host continuation fixture without building a backend. The sixth preserves the replay assertions after rebasing onto the repository-wide Node test-infrastructure consolidation.

Verification

  • Observed RED then GREEN for same connectionId + modelId but different provider-state identities: Anthropic reasoning/signature is removed while portable text remains.
  • Verified real RuntimePolicy mutations: endpoint move, API-key rotation, and request-header credential addition each change providerStateIdentity.
  • Verified one prepared Host activation resolves provider state once, persists that identity for admission, and builds from the same resolved target without a second provider-state read.
  • Verified real SessionManager.resumeSafeBoundaryContinuation() composition carries the same identity through source Run headers, v2 plan/digest, target claim/Run, backend input, and the following normal Run.
  • Verified persisted v1 continuation rows survive SQLite schema migration; v2 rows are accepted; unknown versions are rejected; v1 replay is explicitly unsupported under the v2 projection.
  • Rebased exact-head verification passed: the full SessionManager suite (171 tests), provider/continuation/overflow suites (92 tests), SQLite and workspace-version persistence suites (33 tests), and the complete Runtime Host continuation file (8 tests), including the exact CI failure.
  • Typecheck passed for @maka/runtime, @maka/storage, and @maka/runtime-host.
  • Biome check passed on all currently affected files; git diff --check passed.
  • package-lock.json is unchanged.

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Codex implemented and verified the runtime contract using debug, TDD, adversarial review-feedback, and simplification-audit workflows. Material commits include a Generated-by: Codex trailer.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

@github-actions github-actions Bot added the effort/L Under 1000 readable lines label Aug 30, 2026
@Astro-Han
Astro-Han force-pushed the fix/reasoning-replay-provenance branch 4 times, most recently from a8d4bcb to 1feb2ef Compare August 31, 2026 01:54
@github-actions github-actions Bot added effort/XL Over 1000 readable lines and removed effort/L Under 1000 readable lines labels Aug 31, 2026
@Astro-Han
Astro-Han marked this pull request as ready for review August 31, 2026 09:15

@jackwener jackwener left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the latest head 1715b59c2. No P0 or P1 in the logic — approving the content. No inline comments. Two gate caveats below, and the second one matters for how much this approval is worth.

The admission model is correctly fail-closed, which is the property that counts here. compatibleProviderReasoningReplayEventIds starts from an empty set (plus the current run) and only adds runs that match on both llmConnectionId and modelId. Reasoning from a different connection or a different model is therefore never admitted. Crucially, when targetConnectionId or runHeaders is missing the loop is skipped entirely, so absent inputs produce a smaller allowlist rather than a wider one — missing data cannot silently widen replay. That is the right direction, and it is the opposite of the "optional guard silently inert" pattern I flagged on #4300.

The restriction is also correctly scoped. admitProviderReasoningReplayItems filters only kind === 'thinking', so portable transcript content and tool evidence pass through untouched. It gates provider-owned reasoning specifically rather than trimming history generally.

A note on reading this one commit-by-commit. Intermediate commit 6583c6ebd carries providerReasoningReplayEventIds?: ReadonlySet<string> with a !== undefined escape, which would have been worth flagging. That does not survive to the head: model-history.ts:110 declares it required, and the filter has no undefined branch. I reviewed the head. This is the second of your PRs where an intermediate commit shows a weaker form than the final tree, so it is worth knowing that reviewing by commit here produces findings that no longer exist.

Gate caveat 1: label was still queued; test and windows_recovery are green.

Gate caveat 2, and the more important one: GitHub reports this as CONFLICTING. I confirmed it — merging into current main conflicts, and the single conflicted file is packages/runtime/src/ai-sdk-backend.ts. That is the file carrying the replay-admission wiring I just reviewed: the compatibleProviderReasoningReplayEventIds call site and the threading of the resulting set through the replay and compaction paths. So this approval is bound to this tree, not to whatever comes out of the conflict resolution. Resolving it involves choosing between two versions of security-relevant wiring, which is a judgement call for you rather than something I should do to your branch unasked — but I am happy to rebase it if you want that. Either way the admission wiring is worth re-confirming on the rebased head, and I will re-review on request.

Automated review notice: This comment was posted by an automated review agent operated by jackwener. It is not an independent human review and does not replace one.

@ARE404 ARE404 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving fix(runtime): gate provider reasoning replay by source model at head 1715b59con the code; noting merge is currently blocked on the branch's DIRTY/CONFLICTING state (GitHub reports mergeable: CONFLICTING) so it must be rebased onto current main before it can merge.

I reviewed the gating logic (compatibleProviderReasoningReplayEventIds / admitProviderReasoningReplayItems) and the digest binding:

  • Reasoning-replay is now admitted only when the source run's llmConnectionId+modelId match the target route (or the reasoning is from the current run), which correctly prevents a provider's reasoning being replayed/trusted under a different model connection. The providerReplayDigest binding the admission route prevents a plan built for one model being silently replayed for another.
  • test is green on this exact head (run 33374629392, 24m23s); 0 unresolved review threads.
  • One [P2] note is inline (continuation-replay.ts:129) about PROVIDER_REPLAY_PROJECTION_VERSION staying at 1 despite the changed plan/digest semantics — verify the replay plan isn't durably persisted across upgrades, or bump the version with the change.

No P0–P1.

简体中文

批准 fix(runtime): gate provider reasoning replay by source model,head 1715b59c——基于代码批准;同时注明合并目前被 DIRTY/CONFLICTING 状态阻塞(GitHub 报 mergeable: CONFLICTING),必须先 rebase 到当前 main 才能合。
审查了 gating 逻辑(compatibleProviderReasoningReplayEventIds/admitProviderReasoningReplayItems)与摘要绑定:reasoning-replay 现在仅当源 run 的 llmConnectionId+modelId 与目标路由一致(或 reasoning 来自当前 run)时才被采纳,正确防止某 provider 的 reasoning 在另一模型连接下被重放/信任;providerReplayDigest 绑定 admission 路由,防止为模型 A 生成的 plan 被静默用于模型 B。test 在 exact head 绿(run 33374629392,24m23s);0 未解决线程。行内一条 [P2](continuation-replay.ts:129):plan/digest 语义变了但 PROVIDER_REPLAY_PROJECTION_VERSION 仍是 1——确认 replay plan 不会跨升级持久化,否则应随语义变更 bump 版本。无 P0–P1。

Comment thread packages/runtime/src/continuation-replay.ts
@Astro-Han
Astro-Han force-pushed the fix/reasoning-replay-provenance branch from 1715b59 to 89a0646 Compare August 31, 2026 09:57

@zhiiw zhiiw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at exact head 1715b59c (verified unchanged at review time; test 24m23s and windows_recovery completed/success on this head).

Not approving yet: the PR is CONFLICTING against current main. git merge-tree shows the conflict is confined to packages/runtime/src/ai-sdk-backend.ts — exactly the file where this PR and the now-merged #4287 both rewired the tool-result path. That rebase is semantic, not textual, so the resulting head needs fresh verification; approve then and this becomes a fast review, because the content below is already verified.

What I verified on this head:

  • The reasoning-replay gate is fail-closed and route-exact: compatibleProviderReasoningReplayEventIds admits historical thinking events only when the source run's header matches the target on immutable llmConnectionId + modelId (current-run events are same-route by construction); with no target connection id, nothing historical qualifies. admitProviderReasoningReplayItems filters only thinking items, so text, tool calls, and tool results survive a rejection.
  • The continuation digest now authenticates the route, not just the items: digestProviderReplayAdmission binds provider_replay_admission_v1 to the target connection/model plus the ordered admitted items — closing the planned-for-B/executed-on-C hole where two incompatible targets produced the same item digest.
  • Codex checkpoints bind immutable identity: connectionSlug is gone from openai_codex_remote_v2 in favor of connectionId, and the validator requires it — legacy slug-bound checkpoints fail validation and fall back to the text-summary/raw-history path rather than replaying opaque provider state.
  • Producer side is per-part now: ModelAdapter lowers every SDK reasoning start to one provider-neutral part boundary, Anthropic redactedData rides providerOptions, and the Copilot route throws unless the plaintext contract is selected.

Executed on a real Windows machine at this head: clean forced rebuild, then 568/568 across model-adapter, history-compact-checkpoint, continuation-replay, runtime-resume, runtime-continuation, ai-sdk-backend, session-manager, overflow-reactive-recovery, and computer-use-provider-protocol.


Automated review notice: This comment was posted by an automated review agent operated by zhiiw. It is not an independent human review and does not replace one.

简体中文

暂不批:PR 对当前 main 是 CONFLICTING——merge-tree 显示冲突集中在 ai-sdk-backend.ts,正是本 PR 与已合并的 #4287 同时改写的工具结果路径。这个 rebase 是语义性的,结果 head 需要重新验证;内容本身我已验完(见下),rebase 后重审会很快。机制核实:推理重放门禁 fail-closed 且按不可变 connectionId+modelId 精确匹配(无目标连接则历史一律不放行),只过滤 thinking 项;continuation digest 现在认证目标路由+有序准入项(堵住 B 计划 C 执行的洞);Codex checkpoint 从可变 slug 改绑不可变 connectionId,旧 slug 形校验失败回落文本恢复路径;生产侧 ModelAdapter 把每个 SDK reasoning start 降为中立 part 边界,Anthropic redactedData 走 providerOptions,Copilot 非 plaintext 契约直接抛。本机真 Windows 干净重建后 568/568。

@hqhq1025 hqhq1025 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head 89a06464e898002325db8668325d0dca04a7237a. One P1 blocks approval: the new provider-reasoning admission key does not change when the connection's endpoint or authentication material is replaced in place. The rebase onto current main is otherwise clean, and the reasoning/compaction integration with the durable Tool Result projection passed local validation.

Validation: clean npm ci, npm run build:test, full workspace typecheck, full Runtime suite (3090 passed / 13 skipped), focused Host compaction tests, changed-file Biome, ASF headers, and git diff --check. One unrelated managed-sandbox Host test failed in this Linux container; the changed portion of that file is confined to the Codex compaction test, which passed. Hosted test was queued and windows_recovery was still running at publication time.

Automated review notice: This comment was posted by an automated review agent operated by hqhq1025. It is not an independent human review and does not replace one.

Comment thread packages/runtime/src/model-history.ts Outdated
@Astro-Han
Astro-Han force-pushed the fix/reasoning-replay-provenance branch 2 times, most recently from 6806d34 to d08c4b1 Compare August 31, 2026 14:16
@Astro-Han
Astro-Han force-pushed the fix/reasoning-replay-provenance branch from d08c4b1 to b92a80a Compare August 31, 2026 14:19

@ARE404 ARE404 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Holding approval at exact head 0a5c4d0e26fc88849953dad31fc8d44ebcec92a0: test is red at this head (the gate rule is approve only if no P0/P1 — a failing self-triggered check is a P1-equivalent).

The test check (run 33396097615) fails in packages/runtime-host on a test that is squarely in this PR's domain:

✖ startup parks a pre-claim continuation whose Client Capability is absent (105.902ms)
  AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value: expected: true

This is in packages/runtime-host/src/__tests__/execution-host-continuation.test.ts (line ~314), and this PR modifies the exact subsystem it exercises (runtime-continuation-admission.ts, continuation-replay.ts) — so it reads as self-caused rather than a stale or unrelated failure. Please rebase to current main (branch is also CONFLICTING) and get test green at the head, then I'll re-run the check and this can go through.

For reference, my earlier APPROVE was at head 1715b59c; that conclusion predates the current head 0a5c4d0e (+ the c14f69a5/0a5c4d0e commits) and cannot be carried forward onto this head where CI is not green. Happy to re-lock and re-approve as soon as test passes on it.

@ARE404 ARE404 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewing at the current head b92a80aa74fc1154ec6c6640c6693ab017e313f2 (the head that fixes the gate failure I previously flagged). No P0/P1 — approving.

Since my earlier hold (was against head 0a5c4d0e, where test was red in packages/runtime-host — "startup parks a pre-claim continuation whose Client Capability is absent"), the author has added:

  • 7e2e0899cd — composes continuation-planning authority correctly in the test fixture: the SessionManager used for the parked/pending pre-claim continuation now registers an 'ai-sdk' backend factory that throws if it is ever built, so the test proves that a pending continuation's setup never eagerly builds a backend. This pins the correct invariant (the red test was a real setup-ordering regression) rather than papering over it.
  • b92a80aa74 — aligns the session-manager replay assertions with node:test (assert.strictEqual/assert.match/assert.doesNotMatch in place of the vitest-style expect matchers), preserving the same semantics exactly (e.g. not.toContain('source-only reasoning')doesNotMatch(/source-only reasoning/)); the provider-replay-gate expectations (no source-only leakage, cross-route replay digest binding) are unchanged.

test and windows_recovery are both green at this head. This confirms the PR's core safety property — reasoning replay is gated to the source provider route (providerReplayDigest bound to admission) and fail-closed on cross-model replay — now holds with a passing suite. Approving on code + green CI.

@Astro-Han
Astro-Han merged commit 6b53667 into apache:main Aug 31, 2026
2 checks passed
abhinav-phi pushed a commit to abhinav-phi/maka that referenced this pull request Sep 1, 2026
…4286)

* fix(runtime): fail closed on cross-model reasoning replay

Generated-by: Codex

* fix(runtime): replay Anthropic redacted thinking

Requires the source-route replay gate established by the preceding commit.

Generated-by: Codex

* fix(runtime): replay Copilot chat reasoning

Requires the source-route replay gate established by the first commit in this series.

Generated-by: Codex

* fix(runtime): bind native checkpoints to connection identity

Generated-by: Codex

* fix(runtime): enforce replay admission on recovery

Generated-by: Codex

* fix(runtime): bind continuation replay admission to target route

Generated-by: Codex

* refactor(runtime): privatize Codex compaction projection

Generated-by: Codex

* fix(runtime): preserve ordered reasoning parts

Generated-by: Codex

* fix(runtime): bind replay admission to target route

Generated-by: Codex

* refactor(runtime): remove segment replay metadata

Generated-by: Codex

* feat(runtime): read provider replay projection v2

Generated-by: Codex

* fix(runtime): bind reasoning replay to provider state

Generated-by: Codex

* fix(runtime): prepare provider target once per activation

Generated-by: Codex

* test(storage): follow runtime schema authority

Generated-by: Codex

* test(runtime-host): compose continuation planning authority

Generated-by: Codex

* test(runtime): align replay assertions with node test

Generated-by: Codex
ggbdpq added a commit to ggbdpq/maka that referenced this pull request Sep 2, 2026
…ation

Audit the seven Recovery and resume documents under apache#3522 against current
main. Four documents needed corrections; three were verified accurate as
written.

Corrections:
- recovery-resolver ADR: the Phase 3 decision fact was implemented as the
  `actions.toolRecovery` envelope (`maka.tool.recovery_decision`,
  protocol `tool_recovery_v1`) committed through the atomic recovery
  bundle transaction, not a `tool_recovery_decided` RuntimeEvent; the
  call+dispatch-without-response row resolves to `indeterminate` with a
  resolution-level `requiresReconciliation` (reconciled operations settle
  as terminal `parked` with `reconcile_*` reasons), not a
  `reconcile_required` status; and the journal projection states are now
  `prepared | outcome_committed | recovery_completed | recovery_parked`.
- extraction ledger and Phase 3-4 design: `PROVIDER_REPLAY_PROJECTION_VERSION`
  was frozen at 1 by PR B and advanced to 2 by apache#4286 (cross-model
  reasoning replay gating).
- resume architecture (en + zh-CN): the prior-state recovery park reason
  is the durable `reconcile_matches_prior_state`; `redo_disabled_pending_cas`
  never became a durable code (the Phase 3-4 design keeps it as a UI
  mapping note only).

Verified accurate with no changes: phase0 crash contract (P0-P11
failpoint table, decision vocabulary, twelve-failpoint harness claim),
phase1 safe-boundary contract (flag, planner gates, lifecycle event
names, host entry points), and the extraction ledger's file inventory,
capability names, and schema milestones.

The paired resume-architecture documents move together and both carry
`last_verified: 2026-09-02`; `translation_status: synced` is preserved.

Refs apache#3522

Generated-by: GLM-5.3-Flash (ZCode)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/XL Over 1000 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants