Skip to content

Commit 0decbcc

Browse files
iamnbutlerclaude
andauthored
fix: refuse actions on a minimized window with a restore hint (#59)
Co-authored-by: Claude <noreply@anthropic.com>
1 parent 5d6654f commit 0decbcc

2 files changed

Lines changed: 24 additions & 0 deletions

File tree

‎docs/tools.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,8 @@ With `ocr: true`, `desktop_inspect` (in either mode) and `desktop_screen` also r
8282

8383
`desktop_system_key` is for shortcuts that belong to macOS rather than a window: Spotlight (Command-Space), the app switcher (Command-Tab), Mission Control (Control-Up), the screenshot tools (Command-Shift-4 and 5) and the next input source (Control-Space). It takes the same keys and modifiers as `desktop_key`, but no snapshot. The keys are posted as ordinary keyboard input, so they reach whatever is frontmost, and they often move focus, activate an app or open system UI. Nothing reads back what a shortcut did, so the result is `unknown` even after delivery; observe with `desktop_screen`, `desktop_apps` or `desktop_windows`. Any chord with Command and W, Q, H or M is refused before anything is sent, including Command-Shift-Q (log out).
8484

85+
An action refused because its window was minimized after inspection reports `WINDOW_MINIMIZED`, with the low-level reason as its `cause`. Restore the window with `desktop_restore`, then inspect it again.
86+
8587
## Not supported
8688

8789
- **Insertion outside text fields.** `desktop_insert` needs an observed focused text control with a readable selection.

‎native/sources/client/actions.swift‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -359,6 +359,7 @@ func nativeAction(_ client: PeekabooBridgeClient, handshake: PeekabooBridgeHands
359359
else {
360360
throw ActionError("Literal insertion returned without its expected exact-window receipt. Inspect before retrying.")
361361
}
362+
await explainMinimized(client, &result, receipt: receipt)
362363
return try JSONEncoder().encode(ActionReply(data: result, target_receipt: receipt))
363364
}
364365
let window = try UIAutomationTarget.ExactWindow(identity: identity, bounds: bounds)
@@ -549,9 +550,30 @@ func nativeAction(_ client: PeekabooBridgeClient, handshake: PeekabooBridgeHands
549550
)
550551
// Unknown completion leaves the host's pending lease in place, including client death or response loss.
551552
}
553+
await explainMinimized(client, &result, receipt: receipt)
552554
return try JSONEncoder().encode(ActionReply(data: result, target_receipt: receipt))
553555
}
554556

557+
/// Replaces a refusal's low-level cause when the target window turns out to be minimized.
558+
///
559+
/// A minimized window can't take input, and Accessibility reads of it fail with raw errors; the check
560+
/// runs only after a refusal, so actions that succeed pay nothing for it.
561+
private func explainMinimized(_ client: PeekabooBridgeClient, _ result: inout ActionResult, receipt: Receipt?) async {
562+
guard result.outcome == "refused", let receipt, let windowID = receipt.window_id,
563+
let inventory = try? await client.listWindowMutationInventory(target: .application("PID:\(receipt.pid)")),
564+
let window = inventory.items.first(where: { $0.windowID == windowID }),
565+
window.mutationIdentity.map({ String($0.ownerProcessStartIdentity) }) == receipt.process_start_identity_decimal,
566+
window.isMinimized
567+
else { return }
568+
result.error = ActionMessage(
569+
code: "WINDOW_MINIMIZED",
570+
message: "The target window is minimized, so nothing was sent to it.",
571+
hint: "Restore it with desktop_restore, using its target from desktop_windows, then inspect it again before acting.",
572+
cause: result.error?.message
573+
)
574+
result.requires_fresh_observation = true
575+
}
576+
555577
/// Drags from one window into another, in the same app or another one.
556578
///
557579
/// Exact-window delivery can't leave its window, so this moves the real pointer with global events, and

0 commit comments

Comments
 (0)