Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
114 changes: 112 additions & 2 deletions core/src/workflow/node_context.ts
Original file line number Diff line number Diff line change
Expand Up @@ -117,9 +117,10 @@ export class NodeContext {
this.actions = opts.actions ?? createEventActions();
// Writes via `ctx.state` accumulate into `actions.stateDelta`, mirroring
// Python's `ctx.state` -> `ctx.actions.state_delta` behaviour.
this._state = new State(
opts.invocationContext.session.state,
this._state = new NodeStateView(
invocationOverlay(opts.invocationContext),
this.actions.stateDelta,
opts.invocationContext.session.state,
);
}

Expand Down Expand Up @@ -182,3 +183,112 @@ export class NodeContext {
return executeChildNode({parent: this, node, input, options});
}
}

/**
* Per-invocation record of every `ctx.state` write made by the nodes of one
* invocation, keyed by the session's live state object.
*
* `session.state` object identity is stable for the duration of a turn (the
* session services mutate it in place, and hand out a fresh object per
* `getSession`), so the WeakMap entry naturally scopes to one turn.
*
* The `invocationId` guard covers *sequential* reuse of one state object: a
* later invocation replaces the entry rather than inheriting the earlier one's
* writes. It does not let two *live* invocations share a state object — they
* would evict each other's entry and fall back to reading committed state.
* Nothing does that today: `getSession` hands out a fresh state object per
* turn, and a sub-invocation (`AgentTool`) re-fetches the session, so it keys a
* different entry.
*/
const invocationOverlays = new WeakMap<
Record<string, unknown>,
{invocationId: string; values: Record<string, unknown>}
>();

/** Returns the write overlay for `ic`'s invocation, creating it on first use. */
function invocationOverlay(ic: InvocationContext): Record<string, unknown> {
const sessionState = ic.session.state;
const existing = invocationOverlays.get(sessionState);
if (existing && existing.invocationId === ic.invocationId) {
return existing.values;
}
const created = {invocationId: ic.invocationId, values: {}};
invocationOverlays.set(sessionState, created);
return created.values;
}

/**
* The state view a workflow node sees: its own pending delta, then the
* invocation's write overlay, then the session's committed state.
*
* The overlay exists to keep node-to-node reads honest. `session.state` is
* mutated from two directions during a run: nodes write through it
* immediately, while the runner separately re-applies each event's
* `actions.stateDelta` as it commits that event — and that commit lags node
* execution. Re-applying an earlier node's delta therefore rolls back a later
* node's write, and any node reading in that window observes the stale value:
*
* a: set('attempts', 0)
* b: get -> 0, set('attempts', 1)
* commit(a) re-applies attempts=0 <- rolls back b's write
* c: get -> 0 <- wrong; b already set 1
* commit(b) re-applies attempts=1 <- rolls forward, too late
*
* Reads are served from the overlay, which only ever moves forward, so `c`
* sees `1`. Writes still land in `session.state` as well, so consumers that
* read it directly — notably `{key}` instruction templating in an agent node —
* behave exactly as before.
*
* Two gaps remain. They are mirror images of each other, share the root cause
* above, and are both only really fixable at the source (version-stamping keys
* as the session services apply a delta), so neither is closed here:
Comment on lines +242 to +244

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a nit. A third path misses the overlay: llm_agent_wrapper.ts:141 writes the delta directly.

if (agent.outputKey && output !== undefined) {
  ctx.actions.stateDelta[agent.outputKey] = output;
}

That write skips the overlay and skips session.state, so the next node reads undefined until the event commits. Use ctx.state.set(agent.outputKey, output), which writes the same delta key, plus the overlay and the session. The fault is older than this PR, and no workflow test covers outputKey. llm_agent.ts:708 repeats the pattern outside this package.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5cc73fc — and you undersold it: that write does not just miss the overlay, it lands nowhere at all.

ctx.actions.stateDelta is only ever drained onto an event by FunctionNode.toEvent (function_node.ts:182, via pendingStateDelta). BaseNode.toEvent doesn't, enrichEvent doesn't, and nothing constructs a NodeContext with a shared actions, so for this node the object is write-only. Task-mode outputKey therefore reached neither the overlay, nor session.state, nor committed state — it was dropped, not merely late.

Measured with the new test, which asserts all three (later node reads it, it is on session.state, it is in the committed session):

wrapper writes later node session.state committed
ctx.actions.stateDelta[k] = v (before) ✗ undefined
ctx.state.set(k, v) (as suggested) ✗ undefined
both (landed)

So it needs the write-through and the event stamp:

ctx.state.set(agent.outputKey, output);
event.actions.stateDelta[agent.outputKey] = output;

Test: an agent node's outputKey is visible to the next node — a task-mode LlmAgent over a mock model that calls finish_task, feeding a node that reads the key. First workflow coverage of outputKey, as you said.

On llm_agent.ts:708: left alone deliberately, and it is not the same severity. I probed a single_turn agent node with an outputKey followed by a reader node — the reader does see the value today, because the runner commits the agent's event before the next node starts. That path is exposed to the commit-lag window (the first residual in the NodeStateView doc comment), not to a dropped write, and closing it properly is the version-stamping change rather than a one-liner here.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a nit. I retract two parts of my comment above; your framing is correct.

The value never committed, so "reads undefined until the event commits" was wrong. Nothing drains ctx.actions for this node:

  • BaseNode.toEvent never reads it (base_node.ts:191).
  • enrichEvent only writes agentState (node_runner.ts:250).
  • No caller passes actions to a NodeContext.

ctx.state.set alone was also incomplete — committed state would stay empty. The landed pair is right, and the new test pins all three effects.

*
* - Workflow writes still reach an outside reader out of order. `session.state`
* is written through *and* re-applied by the event commit, so an agent
* resolving `{key}` for a key two nodes wrote can observe the stale value
* inside the same window.
* - Outside writes no longer reach a workflow reader. Once a node writes `k`,
* every later read of `k` in this invocation is served from the overlay, so a
* tool or callback that writes `k` straight to `session.state` mid-run is
* invisible to those reads for the rest of the invocation. Before the overlay
* they saw it, subject to the rollback race being fixed.
*/
class NodeStateView extends State {
constructor(
overlay: Record<string, unknown>,
delta: Record<string, unknown>,
private readonly committed: Record<string, unknown>,
) {
super(overlay, delta);
}

override get<T>(key: string, defaultValue?: T): T | undefined {
if (super.has(key)) {
return super.get<T>(key, defaultValue);
}
return key in this.committed ? (this.committed[key] as T) : defaultValue;
}
Comment on lines +265 to +270

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit. The overlay also shadows writes coming the other way, which is worth naming next to the known residual.

Once a node writes k, every later read of k in this invocation is served from the overlay:

override get<T>(key: string, defaultValue?: T): T | undefined {
  if (super.has(key)) {
    return super.get<T>(key, defaultValue);
  }
  return key in this.committed ? (this.committed[key] as T) : defaultValue;
}

A tool or callback that writes the same key straight to session.state mid-run is therefore invisible to later nodes, for the rest of the invocation. Before the overlay they saw it, subject to the race you are fixing.

This is the mirror of the residual you already document: that one is workflow writes not reaching an outside reader, this one is outside writes not reaching a workflow reader. Same root cause, same real fix (version-stamping). Worth a sentence in the same place, so the next person does not have to derive it.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, and you framed it better than I would have. Documented both directions together in the NodeStateView doc comment (a00c183) rather than only the one I'd noticed:

 * Two gaps remain. They are mirror images of each other, share the root cause
 * above, and are both only really fixable at the source (version-stamping keys
 * as the session services apply a delta), so neither is closed here:
 *
 * - Workflow writes still reach an outside reader out of order. ...
 * - Outside writes no longer reach a workflow reader. Once a node writes `k`,
 *   every later read of `k` in this invocation is served from the overlay, so a
 *   tool or callback that writes `k` straight to `session.state` mid-run is
 *   invisible to those reads for the rest of the invocation. Before the overlay
 *   they saw it, subject to the rollback race being fixed.

Worth stating plainly that this second one is a behaviour change, not just an untouched gap: before the overlay those reads did see the outside write (racily). I think it's the right trade — a mid-run outside writer to a key the workflow also owns is already in undefined-ordering territory, whereas node-to-node read-modify-write is the documented pattern — but it is a trade, and the doc now says so. I'll also carry both into the PR description's Known residual section.


override has(key: string): boolean {
return super.has(key) || key in this.committed;
}

// Both of `State`'s write paths (`set` and `update`) keep writing through to
// the session, so readers of `session.state` (agent instruction templates,
// callbacks, tools) see the write immediately, as they did before the overlay
// existed.

override set(key: string, value: unknown): void {
super.set(key, value);
this.committed[key] = value;
}
Comment on lines +281 to +284

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a nit. State has two write paths and this overrides one of them.

override set(key: string, value: unknown): void {
  super.set(key, value);
  this.committed[key] = value;
}

update() is the other one (state.ts:73), and it is public on ctx.state:

update(delta: Record<string, unknown>) {
  Object.assign(this.delta, delta);
  Object.assign(this.value, delta);   // overlay only
}

So ctx.state.update({...}) lands in the overlay and the node delta but never in session.state — the write-through your set comment protects, and which "writes remain visible on session.state for instruction templating" tests. That test passes because it uses set.

Nothing calls update on a NodeContext today, so this is latent. One line closes it:

override update(delta: Record<string, unknown>): void {
  super.update(delta);
  Object.assign(this.committed, delta);
}

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — fixed in a00c183. Added the override exactly as you wrote it:

override update(delta: Record<string, unknown>): void {
  super.update(delta);
  Object.assign(this.committed, delta);
}

Also moved the write-through comment above both overrides so it reads as a property of NodeStateView, not of set specifically — that framing is what let the gap through in the first place.

Pinned with a test (writes made through update() are visible the same way as set()) asserting both halves of the contract: a later node reads it back, and it lands on session.state. Verified it fails without the override (expected undefined to be 'oceans').


override update(delta: Record<string, unknown>): void {
super.update(delta);
Object.assign(this.committed, delta);
}

override toRecord(): Record<string, unknown> {
return {...this.committed, ...super.toRecord()};
}
}
9 changes: 8 additions & 1 deletion core/src/workflow/nodes/llm_agent_wrapper.ts
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,14 @@ export class LLMAgentWrapper extends BaseNode {
event.output = output;
event.nodeInfo = {...(event.nodeInfo ?? {}), messageAsOutput: true};
if (agent.outputKey && output !== undefined) {
ctx.actions.stateDelta[agent.outputKey] = output;
// Route the write through `ctx.state` so it lands in the invocation
// overlay and on `session.state` right away — a downstream node
// reading `outputKey` runs before this event is committed, and used
// to see `undefined`. Also stamp it on the event being emitted:
// unlike `FunctionNode`, this node never drains `ctx.actions` onto an
// event, so the event delta is what actually gets committed.
ctx.state.set(agent.outputKey, output);
event.actions.stateDelta[agent.outputKey] = output;
}
yield event;
return;
Expand Down
Loading
Loading