fix: restore test parity — no-progress steering regression, stale golden values, midnight-flaky meetings tests#4485
Conversation
…om_raw Leftover from a refactor — the value was never read (the function uses `config` directly downstream), producing an unused-variable warning after recent merges. No behavior change. Seeds the test-parity branch; further fixes follow from CI results. Claude-Session: https://claude.ai/code/session_014RLnG2QbdL3n9TLtfomdhB
…night The test built a meeting at `NOW + 1h` and asserted a "Today" date-group separator. Within an hour of local midnight that offset rolls into the next calendar day, so the meeting lands in the "Tomorrow" bucket and the "Today" separator never renders. CI hit this at 23:14 UTC (Test run 28687331353). Anchor the meeting to noon today instead — it always shares today's local day key regardless of wall-clock time. The component's day-grouping is correct; only the fixture was time-of-day dependent. No behavior change. Claude-Session: https://claude.ai/code/session_014RLnG2QbdL3n9TLtfomdhB
…teering Six Rust test failures surfaced by clean CI (merged without test parity): - middleware (production regression, tinyhumansai#4473): the no-progress ladder's Nudge sent SteeringCommand::Redirect, which is NOT in the Interactive steering allowlist (InjectMessage + Pause only). Every interactive turn where a tool failed twice with identical args or four times with varied errors aborted the whole turn with a Steering error instead of nudging then halting gracefully. Switch the nudge to InjectMessage(system) — equivalent (append + Continue) but within the interactive policy. Fixes the three agent *_raw_coverage_e2e panics (turn_xml_failures…, bus_turn_halts_on_repeated_tool_error…, no_progress_guard_uses_default_iteration_fallback_when_zero). - config schema catalog test: privacy-mode controllers (config_get/set_privacy_mode, added by tinyhumansai#4435/tinyhumansai#4446) were registered but the hand-maintained golden list in config_auth_app_state_connectivity_e2e.rs wasn't updated. Add the two entries. - api::config backend_url test: tinyhumansai#4153 intentionally made a bare `/v1` base on an unknown host classify as an OpenAI-compatible inference base (with its own passing sibling test); the older contradictory assertion wasn't updated. Align it to expect the fallback. - tinyagents middleware inventory tests: tinyhumansai#4444 added MemoryProtocolMiddleware (+1) and tinyhumansai#4473 removed CacheAlignMiddleware (-1), but the count literals were left at 13/11. Correct to 12/10 and drop cache-align from the comment. Claude-Session: https://claude.ai/code/session_014RLnG2QbdL3n9TLtfomdhB
…(UTC)
Same class of bug as the UpcomingTable fix: the shared todayCall/yesterdayCall
fixtures used `NOW - 1h` / `NOW - 25h`, which cross into the previous UTC day
when the suite runs just after 00:00 UTC (CI hit this at ~00:08 UTC) — the
"Today" group then vanished and `getByText('Today')` failed. HistorySection
buckets by UTC calendar day (`utcDayKey`), so anchor the fixtures to noon UTC of
their respective days, making grouping deterministic regardless of run time.
Claude-Session: https://claude.ai/code/session_014RLnG2QbdL3n9TLtfomdhB
…enial assertion
`turn_xml_failures_...` asserts the transcript surfaces a policy denial
("denied by policy 'round17-deny'"). The test set tool_result_budget_bytes: 96;
before the tinyagents 1.5 migration (tinyhumansai#4473) policy denials bypassed the
per-result budget, but the migration now routes them through
ToolOutputMiddleware.after_tool, so 96 bytes truncated the ~400-byte denial to a
stub and the assertion no longer matched. (This assertion was unreachable until
the steering fix in this branch let the turn run to completion.)
Raise the budget above the denial size — no assertion here depends on truncation
actually happening, and production's default budget is 16 KiB so real denials
are never truncated. Documented the underlying regression (denials should be
exempt from the budget) + the dead hard_reject fast-path as a follow-up in code.
Claude-Session: https://claude.ai/code/session_014RLnG2QbdL3n9TLtfomdhB
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 9 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughThis PR makes several unrelated test and behavior adjustments: it stabilizes meeting test fixtures using fixed UTC timestamps, updates backend URL fallback classification test expectations, removes an unused variable in memory sync rebuild, switches a middleware's Nudge escalation to inject a system message instead of redirecting, updates related middleware inventory test counts, raises a tool-result budget in an e2e test, and adds privacy-mode RPC methods to an exposure test. ChangesMeeting Test Determinism
Backend URL Fallback Test
Memory Sync Cleanup
Middleware Nudge Behavior Change
E2E Tool Result Budget
Privacy Mode RPC Exposure Test
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant Tool
participant RepeatedToolFailureMiddleware
participant AgentSession
Tool->>RepeatedToolFailureMiddleware: after_tool result
RepeatedToolFailureMiddleware->>RepeatedToolFailureMiddleware: detect NoProgress::Nudge(instruction)
RepeatedToolFailureMiddleware->>AgentSession: SteeringCommand::InjectMessage(system instruction)
Possibly related issues
Possibly related PRs
Suggested labels: Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Comment |
coderabbitai
Bot
added
agent
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
# Conflicts: # src/openhuman/tinyagents/middleware.rs # src/openhuman/tinyagents/tests.rs
…rge) Two tests came in red from upstream/main via the merge (upstream main is itself red on these — its own PR CI fails on Rust Core Coverage); our changes don't touch the code they exercise: - tinyagents adapter_inventory_registers_model_tools_and_middleware: recent upstream middleware-stack churn produces 13 lifecycle middleware, but the literal still said 14 (its own enumeration lists 13). Match the code. - monitor_agent_e2e orchestrator_gets_denial_when_monitor_command_violates_policy: tinyhumansai#4478 reworded the security-policy denial to "[policy-blocked] Tool '<tool>' was blocked by the security policy"; the test still asserted the old contiguous "[policy-blocked] Security policy". Update the golden substring to the new wording. (ops_install::non_2xx_install_fetch_returns_err_for_4xx_and_5xx also flaked here — a pre-existing env-var-race under llvm-cov, untouched by this branch.) Claude-Session: https://claude.ai/code/session_014RLnG2QbdL3n9TLtfomdhB
1 participant
Summary
main: several PRs merged without updating their tests, plus two clock-dependent flakes. This fixes every genuine, environment-independent failure surfaced by a clean CI run.SteeringCommand::Redirect, which isn't in the Interactive steering allowlist — so every interactive turn that hit repeated tool failures aborted the whole turn instead of nudging then halting gracefully. Now usesInjectMessage.backend_urlinference-base contract, tinyagents middleware inventory counts).UpcomingTable,HistorySection) and theturn_xmlpolicy-denial assertion.git blameof the offending merge — nothing rubber-stamps broken behavior.Problem
A clean CI run on
mainfailed with 1 frontend test + 7 Rust tests, from unrelated merges that didn't update their tests (classic post-merge drift), plus flakes:RepeatedToolFailureMiddleware's nudge emittedSteeringCommand::Redirect; the Interactive steering allowlist permits onlyInjectMessage+Pause, soapply_pending_steeringreturned aSteeringerror and aborted the turn. Impact: any interactive chat turn where a tool fails twice with identical args (or four times with varied errors) hard-errors instead of nudging → halting gracefully. Broke 3 agent*_raw_coverage_e2etests.config_get/set_privacy_mode(added by feat(privacy): S1 — Privacy Mode config + policy + local-only inference enforcement (#4256) #4435/feat(privacy): Privacy Mode + local-only inference enforcement (#4435) #4446) were registered but missing from the hand-maintained golden list inconfig_auth_app_state_connectivity_e2e.rs.api::configbackend_url: /teams/me/usage non-2xx flood: 500 (TAURI-RUST-BSF) + 404 (TAURI-RUST-8C) — RCA gated on #4058 host telemetry #4153 intentionally made a bare/v1base on an unknown host classify as an OpenAI-compatible inference base (with its own passing sibling test); an older, now-contradictory assertion wasn't updated.MemoryProtocolMiddleware(+1) and feat(agent): tinyagents 1.5 migration wave — vendored SDK, dual-write sessions, goals/todos shadows, journals, middleware dedupe #4473 removedCacheAlignMiddleware(−1); the count literals were left at 13/11.UpcomingTable/HistorySectionbuilt fixtures relative toDate.now()(NOW ± 1h), crossing the day boundary when the suite runs within an hour of midnight (CI hit both, ~23:14 and ~00:08 UTC), so the "Today" group vanished.turn_xml_...:852: a deliberately tinytool_result_budget_bytes: 96truncated a ~400-byte policy-denial message. Pre-feat(agent): tinyagents 1.5 migration wave — vendored SDK, dual-write sessions, goals/todos shadows, journals, middleware dedupe #4473 denials bypassed the per-result budget; the migration routes them throughToolOutputMiddleware.after_tool, so 96 B truncated the denial away. (This assertion was unreachable until the steering fix let the turn complete.)Solution
SteeringCommand::RedirecttoInjectMessage(TaMessage::system(...))insrc/openhuman/tinyagents/middleware.rs— both append a system message andContinue, butInjectMessageis permitted on the Interactive policy. Do not widen the allowlist (that would break the documented steering-safety boundary /steering_policy_tightens_by_run_class).privacy_modemethods to the config golden list; align thebackend_urlassertion to expect the fallback; correct the middleware counts 13/11 → 12/10.UpcomingTable(localDayKey) and UTC noon forHistorySection(utcDayKey), matching each component's day basis.turn_xml: raise the artificial test budget above the denial size (production default is 16 KiB, so real denials are never truncated; no assertion here depends on truncation happening).content_rootbinding inrebuild_tree_from_raw(unused-var warning from a merge).Submission Checklist
Rust Core Coverage,Rust Tauri Coverage, andFrontend Checks(coverage) jobs and thePR CI Gateall passed on the fork run for this branch head.N/A: bug/test-parity fixes only, no feature rows added/removed/renamed.## Related—N/A: no feature-ID surface changed.N/A: no release-cut surface touched.Closes #NNN—N/A: no tracking issue.Impact
ok_calls == 0).Related
POLICY_BLOCKED_MARKERand skip the budget/persist path for it inToolOutputMiddleware.after_tool. That also re-enables the no-progresshard_rejectfast-path, which currently never fires for policy denials because their render omits the marker it greps for. Deferred (negligible real impact: 16 KiB default budget) and left as aTODO(follow-up)inline intests/agent_session_turn_raw_coverage_e2e.rs.AI Authored PR Metadata (required for Codex/Linear PRs)
Linear Issue
Commit & Branch
fix/test-paritySummary by CodeRabbit
New Features
Bug Fixes