iamnbutler · Nov 3, 2026 · Nov 3, 2026
diff --git a/docs/app.md b/docs/app.md
@@ -1,6 +1,6 @@
 # Desktop Tools app
 
-`Desktop Tools.app` (`dev.githubnext.desktop-tools`) is a menu bar host for agents that don't embed the library, such as Claude Code or Codex through the [MCP server](mcp.md). It holds its own Accessibility, Screen Recording and Keyboard input grants, requested from its menu while any is missing, and serves `~/.local/state/desktop-tools/desktop.sock`. It links `desktop-client` there to its signed client. Only one copy runs for each state folder. See [Keeping the socket served](#keeping-the-socket-served). **Show agent cursor**, on by default, draws the agent's own pointer for each click, scroll, drag, hover and press. **Highlight agent's window**, also on by default, outlines the window the agent is observing or acting on and labels it with the agent and app, such as "Claude is using TextEdit". See [input](tools.md#input).
+`Desktop Tools.app` (`dev.githubnext.desktop-tools`) is a menu bar host for agents that don't embed the library, such as Claude Code or Codex through the [MCP server](mcp.md). It holds its own Accessibility, Screen Recording and Keyboard input grants, requested from its menu while any is missing, and serves `~/.local/state/desktop-tools/desktop.sock`. It links `desktop-client` there to its signed client. Only one copy runs for each state folder. See [Keeping the socket served](#keeping-the-socket-served). **Show agent cursor**, on by default, draws each agent's own pointer for each click, scroll, drag, hover and press, so several agents' cursors move independently. Each agent has its own color: orange for Claude, blue for Codex, and for any other agent a color picked from its name, the same every time. **Highlight agent's window**, also on by default, outlines the window the agent is observing or acting on and labels it with the agent and app, such as "Claude is using TextEdit". See [input](tools.md#input).
 
 ## Keeping the socket served
 
@@ -64,6 +64,8 @@ Every desktop change is logged, and so are screen captures and clipboard reads.
 
 The menu shows a **Grant** item only for a permission the app is missing. **Pause All Input** sits just above Quit.
 
+**Always Allow Agent-Supplied Passwords**, off by default, lets `desktop_fill_password` fill a password an agent passes in without asking you each time. Keychain items still need macOS's own approval.
+
 **Pause All Input** stops agents from changing anything until you turn it off again. While paused, the app refuses every click, key, typing, scroll, drag, menu command, app or window change, dialog action and clipboard write before it is sent. The refusal tells the agent to ask you to resume. Agents can still observe: listings, the accessibility tree, screen captures and clipboard reads keep working. So does turning on a Chromium or Electron app's web accessibility before a read, since it only lets the app's web content be read and sends no input (see [tools](tools.md#input)). The menu bar icon becomes a pause sign while input is paused. Only this menu item changes it; no client can pause or resume, and agent input can't reach the menu. Pausing isn't saved, so the app starts resumed.
 
 ## Build

diff --git a/docs/embed.md b/docs/embed.md
@@ -19,6 +19,9 @@
 | `char *desktop_tools_activity(void)` | JSON array of the last 20 operations, newest first: `{ time, agent, pid, operation, app?, window?, outcome, count }`; free with `desktop_tools_free`. With a caller handler, `agent` and `pid` name the verified caller |
 | `void desktop_tools_set_log(const char *path)` | Append each operation listed by `desktop_tools_activity` to `path`, one line each, created readable only by the user; past 1 MiB it moves to `<path>.1`. Pass NULL to stop |
 | `void desktop_tools_log(const char *message)` | Append one line of the app's own to that log, such as when the user pauses input |
+| `void desktop_tools_set_password_autofill(bool)` | Fill passwords that agents supply without asking each time; Keychain items still need macOS's approval |
+| `void desktop_tools_set_confirmation_handlers(ask, withdraw)` | Ask the user, on the main thread, before filling an agent-supplied password. `ask` gets an ID and JSON `{ key, title, message, allow, deny }`; `withdraw` gets the ID of a question nobody answered in time. Without them, agent-supplied passwords are refused unless always allowed |
+| `void desktop_tools_answer_confirmation(uint64_t id, bool allowed)` | Answer one question from `ask`; an ID already answered or timed out is ignored |
 | `void desktop_tools_free(void *)` | Release a status or activity string |
 
 macOS grants Accessibility, Screen Recording, and event synthesis to the embedding app. Each embedding app needs its own grants on each machine. Clients must be signed by the host's team with the exact allowlisted identifier. In turn, a client signed with a team sends nothing, not even its hello, to a socket served by a process that isn't the same user's and signed by that team, and fails with `HOST_UNTRUSTED`; this covers `desktop-tools-client` and the app's `desktop-tools-mcp`. Both checks read the signature of the exact process at the other end of the socket, by its audit token. An unsigned build, such as `swift build` makes for development, skips them, as the host checks its client only when it is signed itself. The agent cursor and window outline draw click-through panels from the GUI process, so turn them on only from an app with a running AppKit main loop. The outline's label names the agent from `DESKTOP_TOOLS_AGENT` in the requesting client's environment, which `createDesktop` sets from its `agent` option; without it the label names the program that runs the client, as approvals do. Claude is orange, Codex blue, and any other agent gets a color fixed by its name; each agent has its own cursor and outline. For agents outside an embedding app, use the [Desktop Tools app](app.md).

diff --git a/docs/live-checks.md b/docs/live-checks.md
@@ -45,6 +45,6 @@ These stay manual, because they are destructive, change system settings, need a
 - **#86 and #92 in other apps:** Numbers, Excel, Mail and Chrome, and a long Finder folder of 300 or more files.
 - **#88 in other apps:** Chrome, read-only native text views and Preview PDFs.
 - **#107 web accessibility in Electron apps:** inspect a Slack, VS Code or Discord window and check that `web_accessibility` is `enabled`, with `web_accessibility_via` `manual`, once per launch and the web content's fields are listed, that the app's window still moves and resizes normally, that the log lists **Turn on web accessibility** once, and that it still runs while input is paused.
-- **#5 background paste into web content (spike, before any patch):** with a WebKit app such as Safari in the background and a textarea focused, answer: (1) is Edit > Paste `AXEnabled`, with and without the host's background key-window preparation; (2) does an Accessibility press of Edit > Paste paste into the focused textarea; (3) must the textarea get `AXFocused` first?
+- **#5 in Safari:** checked only in a `WKWebView` test app. With Safari in the background and a page's text field focused, check that the first `desktop_inspect` lists the field and that `desktop_key` Command-V pastes into it.
 - **#27 Codex:** adding the server to Codex's configuration.
 - Anything that asks for a password or an administrator's approval.
diff --git a/docs/node.md b/docs/node.md
@@ -17,7 +17,7 @@ Each result has:
 
 - `text`: bounded JSON written for a model.
 - `data`: the same final object, parsed.
-- `image`: a base64 JPEG or PNG, when the result has one.
+- `image`: `{ data, mimeType }`, base64 JPEG or PNG bytes, when the result has one.
 - `outcome`, for actions: `completed`, `refused`, or `unknown`.
 
 Each request goes whole to the host, through its `desktop-tools-client`, as one tool call, so the host checks the arguments and shapes the result. One client process serves every call (`desktop-tools-client <socket> --serve`) and is replaced when it exits. Results take the compact shape described in [Result shape](tools.md#result-shape). Pass `verbose: true` in the config, or set `DESKTOP_TOOLS_VERBOSE=1`, for native results whole, with their `target_receipt`. Check the inventory completeness fields before relying on a listing. Image bytes appear only in `image`.

diff --git a/docs/recipes.md b/docs/recipes.md
@@ -20,7 +20,7 @@ If the app has **File > Export as PDF…**, as Preview, Pages, Safari and Notes
 3. inspect the window that holds the panel. Find the **PDF** menu button at the bottom left *(unverified: role `AXMenuButton`, title `PDF`)*.
 4. `desktop_click` the PDF button. The fresh observation lists the open menu's items first.
 5. `desktop_element_action` with `action: "AXPick"` on **Save as PDF** (or `desktop_click` it). A Save panel opens.
-6. `desktop_dialog` with the app target. Expect `is_file_panel` true. Use its resolved window target for the next calls.
+6. `desktop_dialog` with the app target. Expect `is_file_dialog` true. Use its resolved window target for the next calls.
 7. `desktop_dialog_answer` with `path: "/Users/me/Desktop/out.pdf"`. Expect `completed`: the panel reads back the folder and name.
 8. `desktop_dialog_answer` with `button: "Save"`.
 
@@ -78,7 +78,7 @@ An app's **File > Open…** (⌘O), or its "add a project" or "choose folder" bu
 5. `desktop_click` the family row, then the typeface row, each from the latest observation.
 6. `desktop_type` the size field with `text: "18"`, then `desktop_key` `enter`.
 
-**Verify:** inspect the panel and check that the family and typeface rows read as selected (`isSelected`) and that the size field shows the value. The text itself carries no readable font through `desktop_read`.
+**Verify:** inspect the panel and check that the family and typeface rows show `selected: true` and that the size field shows the value. The text itself carries no readable font through `desktop_read`.
 
 **Goes wrong:**
 
@@ -234,7 +234,7 @@ There is no pinch or rotate gesture. Use the app's own command: it acts on an ex
 - `desktop_type` on a web field that isn't focused is refused before any input, with a hint to click it first. Clicking is never done implicitly.
 - Typed web text is at most 1,000 UTF-16 code units with no newlines or tabs.
 - An `unknown` result means the readback didn't match. Autocomplete or input masks can reformat the text, such as phone numbers. Inspect before retrying.
-- Password fields are refused, because password entry is not supported.
+- Password fields refuse `desktop_type` with `SECURE_FIELD_REQUIRES_FILL`; fill them with `desktop_fill_password`, preferably from a Keychain item.
 - An autocomplete popup can take focus after typing. Press `escape` with `desktop_key` before the next click if it covers the next field.
 
 ## Pick from a native select in a browser