fix(cli): keep one broken agent from killing the dev server; surface HITL prompts and errors in adk run - #633
Conversation
… server `loadAgentFromFile`/`loadAgentFromDirectory` caught only `AgentFileLoadingError` and rethrew everything else. An agent that throws while constructing — a workflow with an invalid graph, say — raises a plain `Error`, which escaped the handler, rejected the `Promise.all` in `preloadAgents`, and made every endpoint that lists or resolves agents fail. A single malformed agent therefore took the whole dev server with it: `/list-apps` returned 500 and the UI listed nothing. Record such failures instead of propagating them. Broken agents are skipped with an error log naming the file and the reason, the healthy ones still load, and the failure is reported against the app it belongs to via the new `listLoadFailures()`. `getAgentFile` also used to return undefined for an unknown name, surfacing later as "cannot read properties of undefined"; it now throws either the recorded load error or a not-found error listing the available agents.
The REPL printed only text parts, but an interrupt is carried in a `functionCall` part: `RequestInput.message`, the credential request, and a tool-confirmation hint all live in `functionCall.args`. So when a workflow paused, `adk run` printed nothing and dropped straight back to `[user]: ` with the run blocked — the user had to already know a reply was expected and answer blind. Render the three interrupt kinds (`adk_request_input`, `adk_request_credential`, `adk_request_confirmation`) with their message, payload/response schema, auth scheme or tool hint, and a line saying how to answer. The three duplicated event-printing loops are folded into one `printEvent` helper so the resume and replay paths stay consistent. Exports `REQUEST_INPUT_FUNCTION_CALL_NAME` and `REQUEST_EUC_FUNCTION_CALL_NAME` from `@google/adk` so the CLI matches on the canonical constants rather than hardcoding the wire strings.
Whether a run is paused waiting on a human is not visible in an event's text: it is carried in a `functionCall` part named `adk_request_input`, `adk_request_credential` or `adk_request_confirmation`, with the prompt buried in that call's `args`. Every client that wants to ask "is input required, and what for?" has to know how each of the three kinds encodes its id, message and payload — so each one reimplements the same parsing, or renders nothing. Adds `getUserInputRequests`, `requiresUserInput` and `getPendingUserInputRequests`, which flatten all three encodings into one `UserInputRequest` shape. The id to answer with is normalized (it lives in `functionCall.id`, `args.interruptId` or `args.functionCallId` depending on kind), and a tool-confirmation hint is surfaced as `message` so callers can render any kind without a switch. `getPendingUserInputRequests` resolves requests against later `functionResponse` parts, answering the question a UI actually has — "is this session waiting on me right now?" — rather than "was this event an interrupt?". A re-raised interrupt id (a `rerunOnResume` node raises the same id on each attempt) is reported once, since the user still owes one answer. Also gives the three interrupt names a single definition. They were declared in three places — `adk_request_credential` in both `agents/functions.ts` and `workflow/utils/hitl_utils.ts`, `adk_request_confirmation` in both `agents/functions.ts` and `plugins/security_plugin.ts`, and `adk_request_input` in `hitl_utils.ts` plus a bare literal in `tools/request_input_tool.ts`. All of them now live in `agents/functions.ts`; the other sites import or re-export it, so no public import path changes.
An event's failure is reported on the event itself (`errorCode` / `errorMessage`), not as a text part, so the REPL — which printed only text — swallowed it. A turn that failed a safety filter or hit a model error just ended with no output and no explanation. Errors are now printed to stderr as `[author] error: CODE: message`. Also moves interrupt detection out of the CLI: `renderInterrupt` both recognized the three `adk_request_*` function calls and formatted them, which meant the CLI owned parsing that every other client needs too. Detection now comes from `getUserInputRequests`, leaving `renderUserInputRequest` purely presentational — it only decides the wording and how the user is told to answer. The test file faked `@google/adk` wholesale, so these tests asserted against stand-in constants rather than shipped behavior. It now passes the real module through with `importOriginal` and fakes only the Runner and services, so the interrupt cases exercise the real detection logic.
Cuts comments that restated the code they sat above — a `kind: UserInputKind` field documented as "what is being asked for", a `HOW_TO_ANSWER` map documented as how the user answers, doc blocks whose first line repeated the function name, and an inline note duplicating the class doc a few lines up. Comments carrying rationale that is not recoverable from the code are kept and tightened: why a load failure is recorded rather than thrown, why each interrupt kind stores its id somewhere different, and why a re-raised interrupt id is only reported once. Net -41 comment lines, no behavior change.
AmaadMartin
left a comment
There was a problem hiding this comment.
The load-failure isolation is right, and I verified the risky part of it: getAgentFile now throws where it used to return undefined, and no caller depended on that. All three call sites in adk_api_server.ts and the one in deploy_utils.ts either iterate listAgents() or dereference the result immediately, so the old path produced "cannot read properties of undefined" — this is strictly better. No test asserts the old return.
Four items below. The first two are real: the credential branch handles only one of the two encodings in the tree, and replaying a saved session re-announces answered pauses.
| for (const event of loadedSession.events) { | ||
| await sessionService.appendEvent({session, event}); | ||
| const content = event.content; | ||
| if (content && content.parts?.length) { | ||
| const text = content.parts | ||
| .map((part) => part.text || '') | ||
| .join(''); | ||
| if (text) { | ||
| console.log(`[${event.author}]: ${text}`); | ||
| } | ||
| } | ||
| printEvent(event); | ||
| } |
There was a problem hiding this comment.
Not a nit. Replaying a saved session re-announces pauses the user already answered.
for (const event of loadedSession.events) {
await sessionService.appendEvent({session, event});
printEvent(event);
}printEvent calls getUserInputRequests, which reports what an event asked for, with no knowledge of the reply. So --saved_session_file prints "is waiting for your input" and "Type your reply at the next prompt" for every historical interrupt, then hands over to the live prompt. The user cannot tell which pause is real.
getPendingUserInputRequests(loadedSession.events) answers exactly this, and this PR adds it. Render the transcript text here, then print the pending requests once after the loop.
There was a problem hiding this comment.
Agreed, that was wrong. Fixed in e72da44.
The replay loop now prints transcript text only — printEvent(event, {announcePauses: false}) — and getPendingUserInputRequests(loadedSession.events) is rendered once after the loop, so exactly the pauses no functionResponse ever answered, immediately before the live prompt.
Two tests: a transcript whose interrupt was answered prints no "is waiting" at all; a transcript with one answered and one open interrupt prints exactly one. Both fail against the previous code.
| export { | ||
| getPendingUserInputRequests, | ||
| getUserInputRequests, | ||
| requiresUserInput, | ||
| } from './agents/user_input_request.js'; |
There was a problem hiding this comment.
Nit. Two of these three exports have no caller.
export {
getPendingUserInputRequests,
getUserInputRequests,
requiresUserInput,
} from './agents/user_input_request.js';getUserInputRequests is used by cli_run.ts. requiresUserInput and getPendingUserInputRequests are referenced only by their own definitions and this line. AgentLoader.listLoadFailures() is in the same position: the description says a failure "is reported against the app it belongs to", but no server route calls it, and adk_api_server.ts is not in this diff.
My other comment gives getPendingUserInputRequests a real caller. For the remaining two, either add the caller in this PR or hold them back — a public export is hard to withdraw later.
There was a problem hiding this comment.
Fair. All three have real callers now rather than being held back:
getPendingUserInputRequests— the saved-session replay in your other comment.requiresUserInput—runFromInputFileuses it to warn when a scripted--input_filerun ends blocked on a human. That run has no prompt to answer at, so before this it just stopped mid-workflow with no output explaining why.AgentLoader.listLoadFailures()— served atGET /list-app-errors(name, file path, reason). That is the gap you are pointing at:/list-appsomits a broken agent, so it disappears from the dev UI and the only trace is a server log line./list-appskeeps returningstring[], so the UI client is untouched. Three server tests cover the route.
There was a problem hiding this comment.
Correction to the above: the /list-app-errors endpoint is out again (5113073). Exposing load failures over HTTP is a product decision, not something a bug-fix branch should settle, so it should not ride along here.
That leaves listLoadFailures() where your comment found it — a loader method with no server caller. Keeping it deliberately: it is dev-package API rather than a @google/adk export, so it is far cheaper to withdraw than the core helpers, and the surface it will feed is the route decision above. The behaviour that matters is unchanged either way: getAgentFile rethrows the original error for a named app, and every skipped agent is logged with its file and reason.
The other two callers in that reply stand: getPendingUserInputRequests in the saved-session replay, requiresUserInput in the --input_file warning.
`getUserInputRequests` read the workflow encoding only, so a credential pause raised by the agent/tool auth flow rendered empty: `generateAuthEvent` writes `function_call_id` / `auth_config` while `createAuthRequestEvent` writes camelCase. Normalize credential args with `camelCaseKeys`, as `auth_preprocessor` already does, and cover the snake_case producer with a test built by `generateAuthEvent` itself so the two cannot drift apart. Only credential args are rewritten; the other kinds carry a caller-supplied `payload` whose keys must survive untouched. Also renames `REQUEST_EUC_FUNCTION_CALL_NAME` to `REQUEST_CREDENTIAL_FUNCTION_CALL_NAME` so one string no longer travels under two names. Not a public break: the constant was internal before this branch made it an export.
Replaying `--saved_session_file` printed every historical interrupt as if it were live, so the user could not tell which pause the prompt was actually waiting on. Render the transcript without announcing its pauses, then print `getPendingUserInputRequests` once — the requests no `functionResponse` ever answered. Gives the other two helpers a caller as well: - `requiresUserInput` warns when a scripted `--input_file` run ends blocked on a human, which otherwise just stops with no explanation. - `AgentLoader.listLoadFailures()` is served at `GET /list-app-errors`, so an agent that vanished from `/list-apps` because it threw while constructing can be diagnosed without reading server logs.
Reporting load failures over HTTP is a product decision that does not belong in a bug-fix branch, so take the endpoint back out. Nothing else changes: `AgentLoader.listLoadFailures()` stays, `getAgentFile` still rethrows the original error for a named app, and each skipped agent is still logged with its file and reason.
AmaadMartin
left a comment
There was a problem hiding this comment.
Approved. All four comments are addressed, and I verified each against head 51130736 rather than the replies.
Two of your fixes are better than what I asked for. normalizeArgs rewrites credential args only, because the other kinds carry a caller-supplied payload whose keys must survive — I had said to normalize at the read site, which would have corrupted them. And the new credential test is built by calling generateAuthEvent itself, so the two encodings cannot drift apart; hand-rolling the shape was the actual weakness in the original tests.
One correction to my own comment: I wrote that authConfig and message were both undefined for the agent-auth flow. Only authConfig was a casing problem — generateAuthEvent writes no message key at all, so those pauses still render without one, while the doc on UserInputRequest.message says "Populated for every kind". Cosmetic, and your call.
One thing to coordinate: this raises TEST_EXECUTION_TIMEOUT in app_loader_test.ts to 60000 and #639 raises the same line to 80000. Whoever lands second gets a conflict, and the two disagree on the value.
| /** | ||
| * The agents that failed to load. They are excluded from {@link listAgents}, | ||
| * and {@link getAgentFile} rethrows the original error for one by name. | ||
| */ | ||
| async listLoadFailures(): Promise<AgentLoadFailure[]> { |
There was a problem hiding this comment.
Nit, and the revert was the right call — this is just the leftover.
Taking /list-app-errors out of a bug-fix branch is correct scoping; an HTTP surface for load failures is a product decision. But it puts listLoadFailures() back where my earlier comment found it: a public method with a test and no production caller.
Your own revert message lists the two ways a failure already reaches someone:
getAgentFilestill rethrows the original error for a named app, and each skipped agent is still logged with its file and reason.
Both are true, and neither needs this method. Dropping it — with dev/test/utils/agent_loader_test.ts:756 — costs nothing and it comes back with the endpoint that wants it. Keeping it is fine too; it just wants a line saying it is there for the UI change.
Not blocking either way.
`raisedInterrupt` counted any event with `longRunningToolIds`, but that marks every tool declared `isLongRunning`, not a pause for a human. A run that called one and then finished was recorded as paused, so the next turn kept it and fast-forwarded every node — the replay bug this change set fixes, for exactly those workflows. Test the three interrupt call names instead, via `requiresUserInput` (#633), which also covers `adk_request_confirmation` — until now only reachable through the `longRunningToolIds` clause, and untested. The unit fixtures marked a pause with a bare `longRunningToolIds` and no `adk_request_*` call, which no engine event looks like, so none of them caught this. They now build the interrupt the engine actually emits. Adds the two cases that were missing: a completed run that merely used a long-running tool is dropped, and a run paused on a tool confirmation is kept. Also drops two guards that defend against nothing: `i` only grows, so `Math.min` never changes `currentStart`, and `boundary` is never negative, so the `slice` ternary only chose between the array and a copy of it.
`raisedInterrupt` counted any event with `longRunningToolIds`, but that marks every tool declared `isLongRunning`, not a pause for a human. A run that called one and then finished was recorded as paused, so the next turn kept it and fast-forwarded every node — the replay bug this change set fixes, for exactly those workflows. Test the three interrupt call names instead, via `requiresUserInput` (#633), which also covers `adk_request_confirmation` — until now only reachable through the `longRunningToolIds` clause, and untested. The unit fixtures marked a pause with a bare `longRunningToolIds` and no `adk_request_*` call, which no engine event looks like, so none of them caught this. They now build the interrupt the engine actually emits. Adds the two cases that were missing: a completed run that merely used a long-running tool is dropped, and a run paused on a tool confirmation is kept. Also drops two guards that defend against nothing: `i` only grows, so `Math.min` never changes `currentStart`, and `boundary` is never negative, so the `slice` ternary only chose between the array and a copy of it.
* fix(workflow): don't replay a finished run on the next turn Re-invoking a workflow that already completed did not run it. Every node was fast-forwarded from the previous turn's cached output, so the workflow re-emitted its previous answer and the new user message was ignored: turn 1 "first" -> n1(first), n2 -> "n2:n1:first" turn 2 "second" -> (nothing ran) -> "n2:n1:first" Rehydration exists to resume a run that paused, but `reconstructNodeStates` was handed every event in the session, so a finished run looked exactly like one waiting to be resumed. `google/adk-python` scopes this by invocation id — there a resumed invocation keeps its id. The TypeScript runner mints a fresh invocation id for every turn, so a run that spans a pause covers several invocations and an id filter would discard the outputs resume needs (it breaks the auth-gate, HITL and dynamic resume paths). Runs are delimited by pausing instead: an invocation that raised an interrupt ended paused and belongs to the run in progress; one that emitted node events without raising an interrupt ran to completion and becomes the boundary. `eventsForCurrentRun` applies that at the three places that scan history: the graph rehydration, the dynamic scheduler, and the plain-text resume scan in WorkflowAgent — the last one mattered because a plain-text resume never writes a resolving function response, so a finished run's interrupt stayed "pending" forever and swallowed later messages. Engine-minted events (a RequestInput interrupt) carry no invocation id, so the scan carries the last id seen forward; otherwise such an event opens a phantom run and the boundary lands mid-run. Two fixtures were updated to match how real events look: a hand-built prior event now carries the invocation of the run in progress, and a synthetic pending interrupt now carries the node path the engine always stamps. * fix(workflow): only a real HITL pause keeps a run open `raisedInterrupt` counted any event with `longRunningToolIds`, but that marks every tool declared `isLongRunning`, not a pause for a human. A run that called one and then finished was recorded as paused, so the next turn kept it and fast-forwarded every node — the replay bug this change set fixes, for exactly those workflows. Test the three interrupt call names instead, via `requiresUserInput` (#633), which also covers `adk_request_confirmation` — until now only reachable through the `longRunningToolIds` clause, and untested. The unit fixtures marked a pause with a bare `longRunningToolIds` and no `adk_request_*` call, which no engine event looks like, so none of them caught this. They now build the interrupt the engine actually emits. Adds the two cases that were missing: a completed run that merely used a long-running tool is dropped, and a run paused on a tool confirmation is kept. Also drops two guards that defend against nothing: `i` only grows, so `Math.min` never changes `currentStart`, and `boundary` is never negative, so the `slice` ternary only chose between the array and a copy of it.
…HITL prompts and errors in `adk run` (google#633) * fix(cli): isolate agent load failures so one bad agent can't down the server `loadAgentFromFile`/`loadAgentFromDirectory` caught only `AgentFileLoadingError` and rethrew everything else. An agent that throws while constructing — a workflow with an invalid graph, say — raises a plain `Error`, which escaped the handler, rejected the `Promise.all` in `preloadAgents`, and made every endpoint that lists or resolves agents fail. A single malformed agent therefore took the whole dev server with it: `/list-apps` returned 500 and the UI listed nothing. Record such failures instead of propagating them. Broken agents are skipped with an error log naming the file and the reason, the healthy ones still load, and the failure is reported against the app it belongs to via the new `listLoadFailures()`. `getAgentFile` also used to return undefined for an unknown name, surfacing later as "cannot read properties of undefined"; it now throws either the recorded load error or a not-found error listing the available agents. * fix(cli): show human-in-the-loop prompts in `adk run` The REPL printed only text parts, but an interrupt is carried in a `functionCall` part: `RequestInput.message`, the credential request, and a tool-confirmation hint all live in `functionCall.args`. So when a workflow paused, `adk run` printed nothing and dropped straight back to `[user]: ` with the run blocked — the user had to already know a reply was expected and answer blind. Render the three interrupt kinds (`adk_request_input`, `adk_request_credential`, `adk_request_confirmation`) with their message, payload/response schema, auth scheme or tool hint, and a line saying how to answer. The three duplicated event-printing loops are folded into one `printEvent` helper so the resume and replay paths stay consistent. Exports `REQUEST_INPUT_FUNCTION_CALL_NAME` and `REQUEST_EUC_FUNCTION_CALL_NAME` from `@google/adk` so the CLI matches on the canonical constants rather than hardcoding the wire strings. * feat(core): add helpers to inspect pending requests for user input Whether a run is paused waiting on a human is not visible in an event's text: it is carried in a `functionCall` part named `adk_request_input`, `adk_request_credential` or `adk_request_confirmation`, with the prompt buried in that call's `args`. Every client that wants to ask "is input required, and what for?" has to know how each of the three kinds encodes its id, message and payload — so each one reimplements the same parsing, or renders nothing. Adds `getUserInputRequests`, `requiresUserInput` and `getPendingUserInputRequests`, which flatten all three encodings into one `UserInputRequest` shape. The id to answer with is normalized (it lives in `functionCall.id`, `args.interruptId` or `args.functionCallId` depending on kind), and a tool-confirmation hint is surfaced as `message` so callers can render any kind without a switch. `getPendingUserInputRequests` resolves requests against later `functionResponse` parts, answering the question a UI actually has — "is this session waiting on me right now?" — rather than "was this event an interrupt?". A re-raised interrupt id (a `rerunOnResume` node raises the same id on each attempt) is reported once, since the user still owes one answer. Also gives the three interrupt names a single definition. They were declared in three places — `adk_request_credential` in both `agents/functions.ts` and `workflow/utils/hitl_utils.ts`, `adk_request_confirmation` in both `agents/functions.ts` and `plugins/security_plugin.ts`, and `adk_request_input` in `hitl_utils.ts` plus a bare literal in `tools/request_input_tool.ts`. All of them now live in `agents/functions.ts`; the other sites import or re-export it, so no public import path changes. * fix(cli): report event errors in `adk run` An event's failure is reported on the event itself (`errorCode` / `errorMessage`), not as a text part, so the REPL — which printed only text — swallowed it. A turn that failed a safety filter or hit a model error just ended with no output and no explanation. Errors are now printed to stderr as `[author] error: CODE: message`. Also moves interrupt detection out of the CLI: `renderInterrupt` both recognized the three `adk_request_*` function calls and formatted them, which meant the CLI owned parsing that every other client needs too. Detection now comes from `getUserInputRequests`, leaving `renderUserInputRequest` purely presentational — it only decides the wording and how the user is told to answer. The test file faked `@google/adk` wholesale, so these tests asserted against stand-in constants rather than shipped behavior. It now passes the real module through with `importOriginal` and fakes only the Runner and services, so the interrupt cases exercise the real detection logic. * docs: trim redundant comments Cuts comments that restated the code they sat above — a `kind: UserInputKind` field documented as "what is being asked for", a `HOW_TO_ANSWER` map documented as how the user answers, doc blocks whose first line repeated the function name, and an inline note duplicating the class doc a few lines up. Comments carrying rationale that is not recoverable from the code are kept and tightened: why a load failure is recorded rather than thrown, why each interrupt kind stores its id somewhere different, and why a re-raised interrupt id is only reported once. Net -41 comment lines, no behavior change. * fix(core): read both credential arg encodings for user input requests `getUserInputRequests` read the workflow encoding only, so a credential pause raised by the agent/tool auth flow rendered empty: `generateAuthEvent` writes `function_call_id` / `auth_config` while `createAuthRequestEvent` writes camelCase. Normalize credential args with `camelCaseKeys`, as `auth_preprocessor` already does, and cover the snake_case producer with a test built by `generateAuthEvent` itself so the two cannot drift apart. Only credential args are rewritten; the other kinds carry a caller-supplied `payload` whose keys must survive untouched. Also renames `REQUEST_EUC_FUNCTION_CALL_NAME` to `REQUEST_CREDENTIAL_FUNCTION_CALL_NAME` so one string no longer travels under two names. Not a public break: the constant was internal before this branch made it an export. * fix(cli): don't re-ask pauses a saved session already answered Replaying `--saved_session_file` printed every historical interrupt as if it were live, so the user could not tell which pause the prompt was actually waiting on. Render the transcript without announcing its pauses, then print `getPendingUserInputRequests` once — the requests no `functionResponse` ever answered. Gives the other two helpers a caller as well: - `requiresUserInput` warns when a scripted `--input_file` run ends blocked on a human, which otherwise just stops with no explanation. - `AgentLoader.listLoadFailures()` is served at `GET /list-app-errors`, so an agent that vanished from `/list-apps` because it threw while constructing can be diagnosed without reading server logs. * chore: increase test timeout to reduce flakiness * revert(cli): drop the /list-app-errors endpoint Reporting load failures over HTTP is a product decision that does not belong in a bug-fix branch, so take the endpoint back out. Nothing else changes: `AgentLoader.listLoadFailures()` stays, `getAgentFile` still rethrows the original error for a named app, and each skipped agent is still logged with its file and reason.
* fix(workflow): don't replay a finished run on the next turn Re-invoking a workflow that already completed did not run it. Every node was fast-forwarded from the previous turn's cached output, so the workflow re-emitted its previous answer and the new user message was ignored: turn 1 "first" -> n1(first), n2 -> "n2:n1:first" turn 2 "second" -> (nothing ran) -> "n2:n1:first" Rehydration exists to resume a run that paused, but `reconstructNodeStates` was handed every event in the session, so a finished run looked exactly like one waiting to be resumed. `google/adk-python` scopes this by invocation id — there a resumed invocation keeps its id. The TypeScript runner mints a fresh invocation id for every turn, so a run that spans a pause covers several invocations and an id filter would discard the outputs resume needs (it breaks the auth-gate, HITL and dynamic resume paths). Runs are delimited by pausing instead: an invocation that raised an interrupt ended paused and belongs to the run in progress; one that emitted node events without raising an interrupt ran to completion and becomes the boundary. `eventsForCurrentRun` applies that at the three places that scan history: the graph rehydration, the dynamic scheduler, and the plain-text resume scan in WorkflowAgent — the last one mattered because a plain-text resume never writes a resolving function response, so a finished run's interrupt stayed "pending" forever and swallowed later messages. Engine-minted events (a RequestInput interrupt) carry no invocation id, so the scan carries the last id seen forward; otherwise such an event opens a phantom run and the boundary lands mid-run. Two fixtures were updated to match how real events look: a hand-built prior event now carries the invocation of the run in progress, and a synthetic pending interrupt now carries the node path the engine always stamps. * fix(workflow): only a real HITL pause keeps a run open `raisedInterrupt` counted any event with `longRunningToolIds`, but that marks every tool declared `isLongRunning`, not a pause for a human. A run that called one and then finished was recorded as paused, so the next turn kept it and fast-forwarded every node — the replay bug this change set fixes, for exactly those workflows. Test the three interrupt call names instead, via `requiresUserInput` (google#633), which also covers `adk_request_confirmation` — until now only reachable through the `longRunningToolIds` clause, and untested. The unit fixtures marked a pause with a bare `longRunningToolIds` and no `adk_request_*` call, which no engine event looks like, so none of them caught this. They now build the interrupt the engine actually emits. Adds the two cases that were missing: a completed run that merely used a long-running tool is dropped, and a run paused on a tool confirmation is kept. Also drops two guards that defend against nothing: `i` only grows, so `Math.min` never changes `currentStart`, and `boundary` is never negative, so the `slice` ternary only chose between the array and a copy of it.
) `UnsafeLocalCodeExecutor` relied on `spawn`'s own `timeout` option, which kills the interpreter and then leaves the promise waiting on 'close'. 'close' only fires once every stdio stream is closed, so an interpreter that forked rather than exec'd its work leaves a survivor holding those pipes and 'close' never arrives. The promise never settles, the executor's own timeout branch never runs, and the caller waits forever. No timeout value bounds that wait, which is why raising the harness budget twice (#633, #662) did not stop the Windows flake in #622: the wait is unbounded at 5s, at 30s, and at any other number. Windows is where it shows because `powershell` and `python` there are far likelier to leave a live descendant than `bash`/`python3`. Run the timer here instead and release the read ends along with the kill, so the timeout is enforced rather than merely requested. This is the treatment `LocalEnvironment.execute` already applies for the same reason; the executor never got it. Also prefer an explicit `timedOut` flag over inferring the timeout from the close signal. Windows does not report a terminating signal the way POSIX does, so a killed child can close there with a `null` signal and silently skip the timeout message. The regression test reproduces the CI symptom exactly: on the unfixed executor it fails with `Test timed out in 30000ms`, the same string seen on windows-latest. With the fix it completes in ~0.5s. Bug: #622
Stacked on fix/code-executor-process-teardown. The 30s budget was raised in #662 to stop the Windows flake in #622, on the theory that a cold PowerShell start was blowing Vitest's 5s default. It did not work, because the failure was never slowness: a surviving grandchild held the stdio pipes open and 'close' never fired, so the wait was unbounded and no budget could contain it. The parent commit fixes that at the source. Two reasons to put the budget back rather than leave it: The raised value masks real regressions. A hang now fails in ~0.5s with the executor's own "Code execution timed out" message; under a 30s harness budget it would sit there for 30s and report a harness abort instead, which is what made #622 so hard to read. 30s also collided exactly with the executor's own default timeout of 30s, so even a merely-slow run was a coin flip between the two deadlines. That is why the CI failures alternated between "Test timed out in 30000ms" and "Code execution timed out after 30 seconds". A harness budget has to be strictly greater than the thing under test, never equal. Headroom at the restored default: the whole file runs in ~2.0s locally, and the two slowest cases (1009ms, 510ms) are bounded by their own explicit executor timeouts rather than by interpreter start-up, so they do not get slower on a loaded runner. The real-interpreter cases finish in tens of milliseconds. Leaves tests/integration/app_loader/app_loader_test.ts alone. Its 60s covers a genuine `npm install` per fixture, it did not fail in the runs behind #622, and its pre-#633 value of 40000 sat below the project's own INTEGRATION_TEST_TIMEOUT_MS of 60000 -- restoring it would restore a bug. Bug: #622
…794) Stacked on fix/code-executor-process-teardown. The 30s budget was raised in #662 to stop the Windows flake in #622, on the theory that a cold PowerShell start was blowing Vitest's 5s default. It did not work, because the failure was never slowness: a surviving grandchild held the stdio pipes open and 'close' never fired, so the wait was unbounded and no budget could contain it. The parent commit fixes that at the source. Two reasons to put the budget back rather than leave it: The raised value masks real regressions. A hang now fails in ~0.5s with the executor's own "Code execution timed out" message; under a 30s harness budget it would sit there for 30s and report a harness abort instead, which is what made #622 so hard to read. 30s also collided exactly with the executor's own default timeout of 30s, so even a merely-slow run was a coin flip between the two deadlines. That is why the CI failures alternated between "Test timed out in 30000ms" and "Code execution timed out after 30 seconds". A harness budget has to be strictly greater than the thing under test, never equal. Headroom at the restored default: the whole file runs in ~2.0s locally, and the two slowest cases (1009ms, 510ms) are bounded by their own explicit executor timeouts rather than by interpreter start-up, so they do not get slower on a loaded runner. The real-interpreter cases finish in tens of milliseconds. Leaves tests/integration/app_loader/app_loader_test.ts alone. Its 60s covers a genuine `npm install` per fixture, it did not fail in the runs behind #622, and its pre-#633 value of 40000 sat below the project's own INTEGRATION_TEST_TIMEOUT_MS of 60000 -- restoring it would restore a bug. Bug: #622
Link to Issue or Description of Change
2. Or, if no issue exists, describe the change:
Two devtools bugs found while manually exercising the new workflow engine, plus
the small core API the second one needed. One commit per concern.
Problem 1 — one malformed agent takes down the whole dev server
AgentLoader.loadAgentFromFile/loadAgentFromDirectorycaught onlyAgentFileLoadingErrorand rethrew everything else. ButAgentFileLoadingErrormeans this file is not an agent; an agent that throws while constructing
raises a plain
Error. Workflows validate their graph in the constructor, soone bad graph escaped the handler, rejected the
Promise.allinpreloadAgents, and broke every endpoint that lists or resolves agents.Reproduced with 3 healthy agents + 1 invalid graph in the same directory:
Every app became unreachable and the dev UI listed nothing.
Solution: record load failures instead of propagating them. Broken agents are
skipped with an error log naming the file and the reason, healthy agents still
load, and the failure is reported against the app it belongs to via a new
listLoadFailures().getAgentFilealso returnedundefinedfor an unknown name, surfacing later ascannot read properties of undefined. It now throws either the recorded loaderror or a not-found error listing the available agents.
Problem 2 — human-in-the-loop prompts are invisible in
adk runThe REPL printed only
part.text. An interrupt is carried in afunctionCallpart instead:
RequestInput.message, the credential request, and a toolconfirmation hint all live in
functionCall.args. So when a workflow paused,adk runprinted nothing and dropped back to[user]:with the runblocked — the user had to already know a reply was expected, and answer blind.
Solution: render the three interrupt kinds with their message, payload /
response schema, auth scheme or tool hint, and a line saying how to answer.
The same turn also swallowed errors: a failure is reported on the event
itself (
errorCode/errorMessage), not as a text part, so a turn blocked bya safety filter or a model error simply ended with no output. Errors now print
to stderr as
[author] error: CODE: message.Supporting core change —
UserInputRequesthelpersDetecting a pause is not CLI-specific: any client (the dev UI, an SDK consumer)
that wants to ask "is input required, and what for?" has to know that the three
interrupt kinds each store their id in a different place (
functionCall.id,args.interruptId,args.functionCallId) and their prompt under a differentkey. Left in the CLI, that parsing gets reimplemented per surface — or skipped,
which is exactly how Problem 2 happened.
Adds to
@google/adk:flattening all three encodings into one shape (
kind,interruptId,functionCallName,message,payload,responseSchema,toolName,authConfig). A tool-confirmation hint is surfaced asmessageso a caller canrender any kind without a switch.
getPendingUserInputRequestsresolvesrequests against later
functionResponseparts — the question a UI actuallyhas is "is this session waiting on me right now?", not "was this event an
interrupt?" — and reports a re-raised id once, since a
rerunOnResumenoderaises the same id per attempt but the user still owes one answer.
renderUserInputRequestin the CLI is now formatting only.Also deduplicates the interrupt names. They had three definitions between
them:
adk_request_credentialin bothagents/functions.tsandworkflow/utils/hitl_utils.ts,adk_request_confirmationin bothagents/functions.tsandplugins/security_plugin.ts, andadk_request_inputin
hitl_utils.tsplus a bare literal intools/request_input_tool.ts. All nowlive in
agents/functions.ts; the other sites import or re-export, so nopublic import path changes — verified all three still resolve to the same
values off
@google/adk.Testing Plan
Unit Tests:
npx vitest run --project unit:core --project unit:devThe single failure is
dev/test/cli/cli_create_test.ts— pre-existing andenvironment-dependent (it reads the local gcloud project). It fails identically
on a clean checkout of
main.31 new tests:
core/test/agents/user_input_request_test.ts(15) — each interrupt kindsummarized; id fallback; several requests in one event; ordinary tool calls
and plain text ignored; an interrupt with no id to answer ignored; pending
resolution against responses, partial answers, re-raised ids, and ordering.
dev/test/utils/agent_loader_test.ts(5) — a directory containing an agentthat throws on construction still lists and loads the healthy agents; the
failure is attributed to the right app; requesting the broken app rethrows the
original error; an unknown name reports the available agents.
dev/test/cli/cli_run_test.ts(11) — each interrupt kind renders its prompt;an event error is reported; ordinary and unnamed function calls do not
announce a pause.
Two pre-existing test-harness problems were fixed along the way, both of which
were hiding real gaps:
Manual End-to-End (E2E) Tests:
Every claim above was verified end-to-end against real samples, before and after.
Problem 1 — a directory with healthy agents plus one invalid graph:
Problem 2 — offline HITL sample, no API key needed:
Before this change, the block between the two
[user]:prompts was empty.Credential interrupt, offline,
tests/integration/workflows/auth_api_key:npx tsc --noEmit,npx eslintandnpx prettier --checkare clean.Checklist
Additional context
Commits are split so each lands in the right release-please changelog — the
feat(core)incore/CHANGELOG.md, the twofix(cli)indev/CHANGELOG.md:Three smaller bugs turned up in the same pass and are not addressed here, to
keep this reviewable. Happy to send them separately:
adk runmangles absolute paths —cli_run.tsdoespath.join(process.cwd(), agentPath).web/api_serverget this right viagetAbsolutePath().--save_sessionalways fails: it joins onto the agent file path, producingagent.ts/<id>.session.json->ENOTDIR. Its unit test mockssaveToFile,so CI does not catch it. One-line fix.
@experimentalwarning prints minified class names (Class kC is experimental) under the bundling loader, since the decorator readsconstructor.name.