feat(tools): accept natural web_search tool declarations and emit native result blocks

[njbrake]

Description

Clients that speak a provider's own web-search vocabulary never reached a configured search backend. Claude Code and the Anthropic SDK send web_search_20250305, which passed straight through to a provider that may not serve it.

Adds an opt-in web_search_intercept setting. With it on, Otari also claims the provider-named keywords (bare web_search, any web_search_<suffix>) and runs them against its own backend. Off by default, so an upgrade never silently changes who runs a search, and it requires web_search_url: with nothing to intercept to, the keyword still passes through rather than failing the request.

Three deliberate deviations from the issue, all matching LiteLLM's behaviour after reading its websearch_interception implementation:

1. An OpenAI function named web_search is never claimed. That is the caller's own tool; running it server-side means their handler never fires and they never get back a tool_call to dispatch. LiteLLM excludes this case explicitly for the same reason.
2. Interception is opt-in, not implied by a configured backend. LiteLLM gates it behind a callback; Bifrost does not intercept at all.
3. max_uses is accepted but not enforced. max_tool_iterations bounds the loop; searches per iteration are not 1:1.

/v1/messages now emits server_tool_use + web_search_tool_result blocks so citations render, but only for a caller that declared natively. This reverses a decision documented in docs/tools.md: the result block requires encrypted_content, an Anthropic-signed blob. Rather than forge one, it is sent empty (as LiteLLM does) and both minted block types are stripped off inbound messages, so an echoed turn never ships an unsignable block upstream. Stripping only applies when interception is on, so deployments that never opted in are byte-identical.

Also adds GET /v1/tools plus a "how to call this" card on the dashboard Tools page, and retargets a forced tool_choice when a caller declares web search under its own name.

Verified end to end against a llamacpp provider over /v1/messages, patching only the provider's _acompletion so any-llm's real Messages/Completions bridge ran both ways: the search executed, results reached the model, and the response carried the native blocks with gapless indices in both streaming and non-streaming mode.

PR Type

• New Feature
• Bug Fix
• Refactor
• Documentation
• Infrastructure / CI

Relevant issues

Fixes #488

Checklist

• I understand the code I am submitting.
• I have added or updated tests that cover my change (tests/unit, tests/integration).
• I ran the Definition of Done checks locally (make lint, make typecheck, make test).
• Documentation was updated where necessary.
• If the API contract changed, I regenerated the OpenAPI spec (uv run python scripts/generate_openapi.py).

Two integration tests fail on this branch and fail identically on origin/main: test_error_detail_leakage::test_provider_error_does_not_leak_details and test_streaming_error_event::test_streaming_creation_error_returns_http_error. Both expect a 502 from a live provider call.

AI Usage

• No AI was used.
• AI was used for drafting/refactoring.
• This is fully AI-generated.

AI Model/Tool used:

Claude Opus 5 via Claude Code.

Any additional AI details you'd like to share:

Written by Claude through back-and-forth with @njbrake. The three deviations above and the empty-encrypted_content approach were his calls, made after the agent researched how LiteLLM, Bifrost, and the Vercel AI SDK handle the same problem; the prose is the agent's.

• I am an AI Agent filling out this form (check box if true)

Summary

• Added opt-in interception for provider-native web-search declarations.
• Added Anthropic-native search result blocks and message cleanup.
• Added GET /v1/tools for managed-tool discovery.
• Added configuration, dashboard guidance, Docker Compose support, and documentation.
• Preserved existing behavior when interception or a search backend is not configured.
• Added comprehensive integration, unit, and dashboard tests.

[coderabbitai]

Warning

Review limit reached

@njbrake, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 2 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: d06d2376-ab2c-4577-8ba9-d599343ada41

📥 Commits

Reviewing files that changed from the base of the PR and between c56c168 and e08c8df.

⛔ Files ignored due to path filters (1)

• docs/public/openapi.json is excluded by !docs/public/openapi.json

📒 Files selected for processing (30)

• docs/api-reference.md
• docs/dashboard.md
• docs/public/otari.postman_collection.json
• docs/tools.md
• src/gateway/api/routes/messages.py
• src/gateway/services/_tool_loop.py
• src/gateway/services/mcp_loop.py
• src/gateway/services/mcp_loop_messages.py
• src/gateway/services/mcp_loop_responses.py
• src/gateway/static/dashboard/assets/ActivityPage-CfIPwmDz.js
• src/gateway/static/dashboard/assets/BudgetsPage-DJMj9z5Z.js
• src/gateway/static/dashboard/assets/ConfirmDialog-C-RWVwB6.js
• src/gateway/static/dashboard/assets/DocsPage-D4q8s7aN.js
• src/gateway/static/dashboard/assets/KeysPage-DBP0rqYO.js
• src/gateway/static/dashboard/assets/ModelScopeControl-DQpF54p9.js
• src/gateway/static/dashboard/assets/ModelsPage-X1su5jxo.js
• src/gateway/static/dashboard/assets/OverviewPage-33CntAQu.js
• src/gateway/static/dashboard/assets/ProvidersPage-ooS_k3AK.js
• src/gateway/static/dashboard/assets/RoutingPage-Y2JaRSqz.js
• src/gateway/static/dashboard/assets/SettingsPage-eRH6Zlbl.js
• src/gateway/static/dashboard/assets/TablePagination-CMcmgCgb.js
• src/gateway/static/dashboard/assets/ToolsGuardrailsPage-CkNbc6ta.js
• src/gateway/static/dashboard/assets/UsagePage-9xByG5m8.js
• src/gateway/static/dashboard/assets/UsersPage-D5FDH2kZ.js
• src/gateway/static/dashboard/assets/index-CLcUiuVX.js
• src/gateway/static/dashboard/index.html
• tests/unit/test_mcp_loop_messages.py
• tests/unit/test_messages_minted_block_stripping.py
• web/src/api/hooks.ts
• web/src/api/types.ts

Walkthrough

The gateway adds opt-in provider-native web-search interception, Anthropic-native result blocks, managed-tool discovery at GET /v1/tools, dashboard support, configuration settings, documentation, tests, and refreshed dashboard assets.

Changes

Web-search interception and native results

Layer / File(s)	Summary
Interception and routing src/gateway/api/routes/_tools.py, src/gateway/api/routes/_pipeline.py, src/gateway/api/routes/{chat,messages,responses}.py, src/gateway/core/config.py	Provider-native web-search declarations can be intercepted when enabled and backed by a configured URL. Forced tool choices are retargeted to web_search, and native-output options propagate through tool loops.
Native search result blocks src/gateway/services/{web_search_backend,mcp_loop_messages}.py, src/gateway/api/routes/messages.py	Structured search results are converted into Anthropic-native blocks for synchronous and streaming responses. Gateway-minted blocks are removed from subsequent inbound message histories.
Managed tool discovery src/gateway/api/routes/tools.py, src/gateway/api/main.py, scripts/sdk_codegen/sdk-endpoints.txt	Standalone GET /v1/tools reports managed web-search and code-execution tools, schemas, examples, accepted types, and backend availability.
Dashboard and documentation web/src/api/*, web/src/pages/ToolsGuardrailsPage.*, docs/*, docker-compose.yml	The dashboard exposes interception settings and managed-tool call examples. Documentation and Docker Compose configuration describe the new endpoint and setting.
Validation tests/integration/*, tests/unit/*	Tests cover interception, forwarding, tool-choice rewriting, native blocks, transcript sanitization, discovery authentication and availability, and stale-result clearing.
Dashboard bundle wiring src/gateway/static/dashboard/*	The dashboard documentation and Tools & Guardrails bundles are added, and hashed asset references are updated.

Estimated code review effort: 5 (Critical) | ~120 minutes

Possibly related PRs

• mozilla-ai/otari#144 — Directly overlaps web-search gateway handling and WebSearchBackend behavior.
• mozilla-ai/otari#350 — Shares the configuration, tool-settings, pipeline, and dashboard infrastructure extended here.
• mozilla-ai/otari#504 — Modifies the same web-search and tool-loop execution paths.

Suggested reviewers: tbille

🚥 Pre-merge checks | ✅ 1 | ❌ 4

❌ Failed checks (4 warnings)

Check name	Status	Explanation	Resolution
Title check	⚠️ Warning	The title describes the feature and uses an accepted prefix, but it is 86 characters and exceeds the requested approximately 70-character limit.	Shorten the title to approximately 70 characters or fewer, such as "feat: intercept natural web_search declarations".
Linked Issues check	⚠️ Warning	Most objectives in #488 are implemented, but OpenAI function tools named web_search are preserved instead of intercepted as required by the issue.	Either intercept OpenAI function declarations as required by #488 or update the linked issue acceptance criteria before merging.
Out of Scope Changes check	⚠️ Warning	The added bundled DocsPage is unrelated to the linked web-search and tool-discovery objectives, although the ToolsGuardrails bundle changes are in scope.	Remove the unrelated DocsPage bundle change and associated asset churn, or link an issue that requires it.
Docstring Coverage	⚠️ Warning	Docstring coverage is 6.07% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (1 passed)

Check name	Status	Explanation
Description check	✅ Passed	The description includes the required sections, issue link, checklist, test caveat, documentation details, and AI usage disclosure.

✨ Finishing Touches 💡 2

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/web-search-tool-declarations

✨ Simplify code

• Create PR with simplified code
• Commit simplified code in branch fix/web-search-tool-declarations

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[njbrake]

@coderabbitai review

Note: this comment was posted by Claude via back-and-forth with @njbrake. The reasoning and decisions are his.

[coderabbitai]

@njbrake I will review the changes in #523. I will check the interception behavior, provider compatibility, API contract changes, and test coverage.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

Actionable comments posted: 2

🧹 Nitpick comments (1)

src/gateway/api/routes/_tools.py (1)

9-25: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Replace prohibited prose separators.

Use punctuation or rephrase these comments. Do not use em dashes or double-hyphen separators in Python prose.

• src/gateway/api/routes/_tools.py#L9-L25: Replace the em dash separator.
• src/gateway/services/mcp_loop_messages.py#L172-L183: Replace the em dash in the inline comment.
• tests/unit/test_chat_request_helpers.py#L3-L14: Replace the em dash separator.
• tests/unit/test_chat_request_helpers.py#L164-L164: Replace the hyphen heading separator.
• tests/unit/test_chat_request_helpers.py#L230-L230: Replace the hyphen heading separator.
• tests/unit/test_chat_request_helpers.py#L278-L278: Replace the hyphen heading separator.
• tests/unit/test_mcp_loop_messages.py#L1077-L1082: Replace the hyphen heading separator.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/gateway/api/routes/_tools.py` around lines 9 - 25, Replace prohibited em
dash and hyphen separators in the prose comments and test headings at
src/gateway/api/routes/_tools.py lines 9-25,
src/gateway/services/mcp_loop_messages.py lines 172-183,
tests/unit/test_chat_request_helpers.py lines 3-14, 164, 230, and 278, and
tests/unit/test_mcp_loop_messages.py lines 1077-1082. Rephrase or use standard
punctuation while preserving the existing meaning; no behavioral code changes
are needed.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/gateway/api/routes/messages.py`:
- Around line 154-158: Update the filtering predicate used to build kept_blocks
so it removes only gateway-minted native blocks with unambiguous gateway
provenance, preserving provider-generated server_tool_use and
web_search_tool_result blocks. Add a regression test covering an
enabled-interception request with a provider-generated native pair and verify
the provider receives both blocks unchanged.

In `@src/gateway/services/mcp_loop_messages.py`:
- Around line 443-446: Update the mixed tool-batch paths in
src/gateway/services/mcp_loop_messages.py at lines 443-446 and 647-649 to
collect native blocks whenever a gateway-owned search executes, not only in the
all-owned continuation path; ensure both non-streaming and streaming exits emit
the corresponding server_tool_use and web_search_tool_result blocks, and add
tests covering one gateway search combined with one caller-managed tool in each
mode.

---

Nitpick comments:
In `@src/gateway/api/routes/_tools.py`:
- Around line 9-25: Replace prohibited em dash and hyphen separators in the
prose comments and test headings at src/gateway/api/routes/_tools.py lines 9-25,
src/gateway/services/mcp_loop_messages.py lines 172-183,
tests/unit/test_chat_request_helpers.py lines 3-14, 164, 230, and 278, and
tests/unit/test_mcp_loop_messages.py lines 1077-1082. Rephrase or use standard
punctuation while preserving the existing meaning; no behavioral code changes
are needed.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 9bbbcd47-dd88-4869-8292-20419de6ac4f

📥 Commits

Reviewing files that changed from the base of the PR and between f9c59f7 and c56c168.

⛔ Files ignored due to path filters (1)

• docs/public/openapi.json is excluded by !docs/public/openapi.json

📒 Files selected for processing (52)

• docker-compose.yml
• docs/api-reference.md
• docs/configuration.md
• docs/dashboard.md
• docs/public/otari.postman_collection.json
• docs/tools.md
• scripts/sdk_codegen/sdk-endpoints.txt
• src/gateway/api/main.py
• src/gateway/api/routes/_pipeline.py
• src/gateway/api/routes/_tools.py
• src/gateway/api/routes/chat.py
• src/gateway/api/routes/messages.py
• src/gateway/api/routes/responses.py
• src/gateway/api/routes/settings.py
• src/gateway/api/routes/tool_settings.py
• src/gateway/api/routes/tools.py
• src/gateway/core/config.py
• src/gateway/services/mcp_loop_messages.py
• src/gateway/services/sandbox_backend.py
• src/gateway/services/tool_settings_service.py
• src/gateway/services/web_search_backend.py
• src/gateway/static/dashboard/assets/ActivityPage-CBkPEqTm.js
• src/gateway/static/dashboard/assets/BudgetsPage-CYOXYjZv.js
• src/gateway/static/dashboard/assets/ConfirmDialog-DgcF58qQ.js
• src/gateway/static/dashboard/assets/DocsPage-mZIYniek.js
• src/gateway/static/dashboard/assets/KeysPage-LWzPuWOd.js
• src/gateway/static/dashboard/assets/ModelScopeControl-D0HVPN2y.js
• src/gateway/static/dashboard/assets/ModelsPage-CwFQs9Yf.js
• src/gateway/static/dashboard/assets/OverviewPage-BkCDNMJs.js
• src/gateway/static/dashboard/assets/ProvidersPage-DUovBYoY.js
• src/gateway/static/dashboard/assets/RoutingPage-Cus6VumX.js
• src/gateway/static/dashboard/assets/SettingsPage-N5HYVsA_.js
• src/gateway/static/dashboard/assets/TablePagination-C7a0MSTB.js
• src/gateway/static/dashboard/assets/ToolsGuardrailsPage-DNC3Wa-x.js
• src/gateway/static/dashboard/assets/ToolsGuardrailsPage-tLXMuWh_.js
• src/gateway/static/dashboard/assets/UsagePage-hKpUiHP6.js
• src/gateway/static/dashboard/assets/UsersPage-CUI67MS_.js
• src/gateway/static/dashboard/assets/index-BE7E0N2z.js
• src/gateway/static/dashboard/assets/index-Bywr3Bjy.js
• src/gateway/static/dashboard/index.html
• tests/integration/test_hybrid_mode_messages.py
• tests/integration/test_messages_route_dispatch.py
• tests/integration/test_web_search_interception.py
• tests/unit/test_chat_request_helpers.py
• tests/unit/test_mcp_loop_messages.py
• tests/unit/test_messages_minted_block_stripping.py
• tests/unit/test_tools_endpoint.py
• tests/unit/test_web_search_backend.py
• web/src/api/hooks.ts
• web/src/api/types.ts
• web/src/pages/ToolsGuardrailsPage.test.tsx
• web/src/pages/ToolsGuardrailsPage.tsx

💤 Files with no reviewable changes (2)

• src/gateway/static/dashboard/assets/ToolsGuardrailsPage-DNC3Wa-x.js
• src/gateway/static/dashboard/assets/index-BE7E0N2z.js