Skip to content

claw-mcp contract suite accepts action calls but page state sometimes never changes under CI #2592

Description

Summary

The Tests / claw-mcp CI shard can fail with a cluster of browser-action contract timeouts where MCP action calls return successfully, but the fixture page never observes the expected DOM/input state change.

Observed on PR #2585 in the Tests / claw-mcp job:
https://github.com/browseros-ai/BrowserOS/actions/runs/34223702353/job/102053036980

Observed failures

The failed run reported 97 passing tests and 12 failures. The failures cluster around browser actions and one download timeout:

  • navigate: reload resets in-page state
  • snapshot concurrency: cursor refs remain actionable
  • act: click updates the page
  • act: click_at hits page coordinates
  • act: type enters text into a field
  • act: type_at enters text at coordinates
  • act: fill sets a single field
  • act: press Enter submits the form
  • act: click toggles a checkbox and is repeatable
  • act: check and uncheck kinds set checkbox state
  • run: SDK end-to-end pages.list -> snapshot -> click -> return
  • download: file lands on disk and its path is reported

Example error from the log:

error: timed out after 15000ms waiting for the run script click to update #result

The download case also timed out:

error: download unexpectedly failed: download failed: Download timed out after 60000ms

Why this looks like a test/runtime issue

The associated PR only changed MCP JSON schema normalization and Rust schema tests. Local MCP Rust verification passed:

bun run --filter @browseros/browseros-mcp-rust test:rust
bun run --filter @browseros/browseros-mcp-rust lint:rust

The failing shard is the BrowserOS/AppImage-driven contract suite running under the CI wrapper with BROWSEROS_TEST_HEADLESS=false and xvfb-run. The failure pattern suggests the server/browser connection remains alive and many contracts pass, but input/download effects intermittently do not reach the page.

Expected behavior

If act, run, and download return success in the contract suite, the fixture page should observe the corresponding DOM/input/download side effect within the contract timeout. If the browser action is not delivered, the tool should fail with a diagnostic that identifies the failed dispatch instead of returning success and leaving the test to time out.

Suggested investigation

  • Capture BrowserOS stderr/stdout for failed claw-mcp CI runs, at least on retry/failure.
  • Add action-level tracing around CDP dispatch completion versus DOM event delivery for act calls.
  • Check whether the AppImage + xvfb-run path can lose focus/input events while still reporting successful dispatch.
  • Consider adding failure artifacts/screenshots for the action timeout cases.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions