Skip to content

Commit a08b9cf

Browse files
authored
feat(desktop): close exact windows without retrying native input (#134)
1 parent f9e0db3 commit a08b9cf

9 files changed

Lines changed: 224 additions & 14 deletions

File tree

‎apps/desktop/native/patches/README.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,16 @@ The quit-specific result validator permits this canonical outcome with `false` t
6161
the existing bridge returns the boolean and its signed process receipt. Other false/success
6262
contradictions stay errors. Force-quit behavior, target revalidation, and mutation lanes are unchanged.
6363

64+
`peekaboo-close.patch` makes background window close a single request for
65+
[window management](https://github.com/githubnext/ace2/issues/73). It selects a supported `AXClose`
66+
or exact close-button `AXPress` from read-only evidence before input, then rechecks the original
67+
window ID, process generation and bounds immediately before that one action. No accepted or
68+
ambiguous native result falls through to another close route. An accepted close whose window
69+
remains open is unverified with unsafe retry, including when unsaved work opens a dialog.
70+
Cancellation stops admission before input; once admitted, the existing mutation lane remains
71+
held until the detached AX call actually settles. Confirmed disappearance retains the existing
72+
bridge postcondition checks. Foreground fallback behavior is unchanged and is not exposed by Ace.
73+
6474
`peekaboo-clipboard-text.patch` adds explicit plain-text clipboard reads and persistent writes for
6575
[clipboard access](https://github.com/githubnext/ace2/issues/72). It applies after the insertion patch
6676
and reuses its GUI clipboard service and reservation gate. Reads require silent clipboard access,
Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
diff --git a/Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/UI/WindowManagementService+StateOperations.swift b/Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/UI/WindowManagementService+StateOperations.swift
2+
index cc8ea70e4..d0c57fa06 100644
3+
--- a/Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/UI/WindowManagementService+StateOperations.swift
4+
+++ b/Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/UI/WindowManagementService+StateOperations.swift
5+
@@ -3,6 +3,7 @@ import AppKit
6+
import AXorcist
7+
import CoreGraphics
8+
import Foundation
9+
+import os
10+
import PeekabooFoundation
11+
12+
@MainActor
13+
@@ -70,21 +71,25 @@ extension WindowManagementService {
14+
hint: "Restore the same exact window first, or retry with explicit foreground consent.")
15+
}
16+
17+
- let attempt = try await self.attemptPinnedBackgroundClose(expectedIdentity)
18+
- guard attempt.dispatched else {
19+
- throw OperationError.interactionFailed(
20+
- action: "close window",
21+
- reason: "Window close operation failed")
22+
- }
23+
- guard attempt.disappeared else {
24+
- throw WindowManagementActionOutcome.suspectedNoop(
25+
+ try await BoundedBackgroundWindowAX.dispatchOneClose(expectedIdentity: expectedIdentity)
26+
+ do {
27+
+ guard try await self.verifyPinnedWindowClose(expectedIdentity) == .succeeded else {
28+
+ // An accepted close may be waiting on unsaved work; it is not safe to repeat.
29+
+ return .dispatchedUnverified(
30+
+ delivery: WindowManagementActionOutcome.backgroundActionDelivery,
31+
+ evidence: .operationStillRunning,
32+
+ unitCount: .one)
33+
+ }
34+
+ } catch {
35+
+ throw WindowManagementActionOutcome.dispatchedUnverified(
36+
action: "close window",
37+
delivery: WindowManagementActionOutcome.backgroundActionDelivery,
38+
- dispatchCount: attempt.dispatchCount)
39+
+ dispatchCount: 1,
40+
+ cause: error)
41+
}
42+
- return WindowManagementActionOutcome.confirmedChange(
43+
+ return .confirmedChange(
44+
delivery: WindowManagementActionOutcome.backgroundActionDelivery,
45+
- dispatchCount: attempt.dispatchCount)
46+
+ unitCount: .one)
47+
}
48+
}
49+
return DesktopActionResult(outcome: outcome)
50+
@@ -1087,6 +1092,106 @@ private enum BoundedBackgroundWindowAX {
51+
}
52+
}
53+
54+
+ static func dispatchOneClose(expectedIdentity: WindowMutationIdentity) async throws {
55+
+ let cancelled = OSAllocatedUnfairLock(initialState: false)
56+
+ try await withTaskCancellationHandler {
57+
+ let result: Result<Void, DesktopActionFailure> = await self.perform(expectedIdentity: expectedIdentity) {
58+
+ guard SystemIdentityResolver.validateWindowMutationIdentity(expectedIdentity),
59+
+ let capturedBounds = expectedIdentity.capturedBounds,
60+
+ let windowID = CGWindowID(exactly: expectedIdentity.windowID),
61+
+ let rawWindow = self.exactWindow(
62+
+ windowID: windowID,
63+
+ ownerPID: expectedIdentity.ownerProcessIdentifier)
64+
+ else {
65+
+ return .failure(.preDispatchRefusal(
66+
+ reason: .targetUnavailable,
67+
+ message: "The exact window could not be verified before close."))
68+
+ }
69+
+ return AXChildWindowMessagingTimeout.perform(on: rawWindow, timeout: self.messagingTimeout) { window in
70+
+ var actionsValue: CFArray?
71+
+ guard AXUIElementCopyActionNames(window, &actionsValue) == .success,
72+
+ let actions = actionsValue as? [String]
73+
+ else {
74+
+ return .failure(.preDispatchRefusal(
75+
+ reason: .operationUnsupported,
76+
+ message: "The exact window's supported close actions could not be read."))
77+
+ }
78+
+ // Choose one route from read-only evidence. No native failure selects another route.
79+
+ let receiver: AXUIElement
80+
+ let action: CFString
81+
+ if actions.contains("AXClose") {
82+
+ receiver = window
83+
+ action = "AXClose" as CFString
84+
+ } else {
85+
+ var buttonValue: CFTypeRef?
86+
+ guard AXUIElementCopyAttributeValue(
87+
+ window, kAXCloseButtonAttribute as CFString, &buttonValue) == .success,
88+
+ let buttonValue, CFGetTypeID(buttonValue) == AXUIElementGetTypeID()
89+
+ else {
90+
+ return .failure(.preDispatchRefusal(
91+
+ reason: .operationUnsupported,
92+
+ message: "The exact window exposes no supported background close route."))
93+
+ }
94+
+ receiver = unsafeDowncast(buttonValue, to: AXUIElement.self)
95+
+ action = kAXPressAction as CFString
96+
+ }
97+
+ return AXChildWindowMessagingTimeout.perform(on: receiver, timeout: self.messagingTimeout) { target in
98+
+ if !CFEqual(target, window) {
99+
+ var buttonActionsValue: CFArray?
100+
+ var currentButton: CFTypeRef?
101+
+ guard AXUIElementCopyActionNames(target, &buttonActionsValue) == .success,
102+
+ (buttonActionsValue as? [String])?.contains(kAXPressAction as String) == true,
103+
+ AXUIElementCopyAttributeValue(
104+
+ window, kAXCloseButtonAttribute as CFString, &currentButton) == .success,
105+
+ let currentButton, CFEqual(currentButton, target)
106+
+ else {
107+
+ return .failure(.preDispatchRefusal(
108+
+ reason: .operationUnsupported,
109+
+ message: "The exact window's close button could not be verified."))
110+
+ }
111+
+ }
112+
+ var ownerPID: pid_t = 0
113+
+ var currentWindowID: CGWindowID = 0
114+
+ guard AXUIElementGetPid(target, &ownerPID) == .success,
115+
+ ownerPID == expectedIdentity.ownerProcessIdentifier,
116+
+ AXWindowIDResolver.copyWindowID(window, into: &currentWindowID) == .success,
117+
+ currentWindowID == windowID,
118+
+ self.bounds(of: window) == capturedBounds,
119+
+ self.boolAttribute(kAXMinimizedAttribute as String, of: window) == false,
120+
+ SystemIdentityResolver.validateWindowMutationIdentity(expectedIdentity)
121+
+ else {
122+
+ return .failure(.preDispatchRefusal(
123+
+ reason: .targetUnavailable,
124+
+ message: "The exact window changed before close."))
125+
+ }
126+
+ guard !cancelled.withLock({ $0 }) else {
127+
+ return .failure(.preDispatchRefusal(
128+
+ reason: .requestCancelled,
129+
+ message: "Close was cancelled before native dispatch."))
130+
+ }
131+
+ let nativeResult = AXUIElementPerformAction(target, action)
132+
+ if nativeResult == .success { return .success(()) }
133+
+ let error = ActionInputDriver.classify(nativeResult)
134+
+ if ActionInputDriver.nativeMutationFailureMayHaveDispatched(error) {
135+
+ return .failure(.indeterminate(
136+
+ delivery: WindowManagementActionOutcome.backgroundActionDelivery,
137+
+ evidence: .completionUnknown,
138+
+ unitCount: .one,
139+
+ message: "The close request may have been delivered; native completion is unknown.",
140+
+ hint: "Refresh the window inventory; do not blindly retry close.",
141+
+ causeDescription: String(describing: error)))
142+
+ }
143+
+ return .failure(WindowManagementActionOutcome.refused(action: "close window", error: error))
144+
+ }
145+
+ }
146+
+ }
147+
+ // Keep the coordinator lane until the actual AX call settles, including after cancellation.
148+
+ try result.get()
149+
+ } onCancel: {
150+
+ cancelled.withLock { $0 = true }
151+
+ }
152+
+ }
153+
+
154+
static func dispatchClose(
155+
expectedIdentity: WindowMutationIdentity,
156+
action: BoundedBackgroundWindowCloseAction) async -> Bool

‎apps/desktop/native/sources/client/management.swift‎

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ struct ManagementTarget: Codable {
5858

5959
private struct ManagementRequest: Decodable {
6060
enum Operation: String, Decodable {
61-
case activate, quit, focus, minimize, restore, move, resize
61+
case activate, quit, close, focus, minimize, restore, move, resize
6262
}
6363
struct Position: Decodable {
6464
let x: Double
@@ -123,6 +123,7 @@ func nativeManagement(_ client: PeekabooBridgeClient, handshake: PeekabooBridgeH
123123
var result = ManagementResult()
124124
var receipt: Receipt?
125125
var invoked = false
126+
var closing: WindowMutationIdentity?
126127
do {
127128
let request = try readManagement()
128129
result.action = request.op.rawValue
@@ -184,6 +185,12 @@ func nativeManagement(_ client: PeekabooBridgeClient, handshake: PeekabooBridgeH
184185
), supportsPinnedQuit: true)
185186
terminated = action.payload
186187
outcome = action.outcome
188+
case .close:
189+
operation = .backgroundCloseWindow
190+
closing = window
191+
outcome = try await client.closeWindowResult(
192+
target: .windowId(window!.windowID), expectedIdentity: window!, allowForegroundFallback: false
193+
).outcome
187194
case .focus:
188195
operation = .focusWindow
189196
let action = try await client.focusWindowResult(target: .windowId(window!.windowID), expectedIdentity: window!)
@@ -231,17 +238,24 @@ func nativeManagement(_ client: PeekabooBridgeClient, handshake: PeekabooBridgeH
231238
if terminated == false {
232239
result.message = "The normal quit request was accepted, but termination was not confirmed. Inspect remaining windows for unsaved work or other dialogs; do not blindly retry or force quit."
233240
}
241+
if request.op == .close, !outcome.isConfirmed {
242+
result.message = "The close request was not confirmed complete. Inspect remaining windows for unsaved work or other dialogs; do not blindly retry close."
243+
}
234244
switch outcome.state {
235245
case .refused:
236246
result.outcome = "refused"
237247
case .indeterminate, .partial:
238248
result.outcome = "unknown"
239249
default:
240-
result.outcome = outcome.evidence == .operationStillRunning || terminated == false ? "unknown" : "completed"
250+
result.outcome = outcome.evidence == .operationStillRunning || terminated == false || (request.op == .close && !outcome.isConfirmed) ? "unknown" : "completed"
241251
}
242252
} catch let failure as DesktopActionFailure {
243253
result.outcome = failure.outcome.dispatchState.mutationDispatched ? "unknown" : "refused"
244254
result.native_outcome = failure.outcome
255+
// The bridge attributes failures only after verifying the signed request-bound target.
256+
if let closing, failure.targetReceipt == closing.actionTargetReceipt {
257+
receipt = Receipt(pid: closing.ownerProcessIdentifier, window_id: closing.windowID, process_start_identity_decimal: String(closing.ownerProcessStartIdentity))
258+
}
245259
result.requires_fresh_observation = failure.outcome.dispatchState.mutationDispatched || failure.outcome.escalation == .refreshTarget
246260
result.error = ManagementMessage(code: failure.standardErrorCode?.rawValue ?? "DESKTOP_ACTION_FAILED", message: failure.message, hint: failure.hint)
247261
} catch {

‎apps/desktop/native/sources/desktop/desktop.swift‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ private final class Desktop {
7171
.ownsSnapshot, .getDetectionResult, .beginSnapshotMutation, .finishSnapshotMutation,
7272
.targetedClick, .exactWindowTargetedClick, .setValue, .exactWindowTargetedHotkey,
7373
.selectText, .literalInsert, .clipboardTextRead, .clipboardTextWrite, .targetedScroll, .exactWindowDrag,
74-
.activateApplication, .quitApplication, .focusWindow, .minimizeWindow, .restoreWindow,
74+
.activateApplication, .quitApplication, .backgroundCloseWindow, .focusWindow, .minimizeWindow, .restoreWindow,
7575
.moveWindow, .resizeWindow,
7676
],
7777
hostKind: .gui,

‎apps/desktop/scripts/helper.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -154,6 +154,7 @@ const patches = [
154154
"peekaboo-insert.patch",
155155
"peekaboo-pointer-window.patch",
156156
"peekaboo-quit.patch",
157+
"peekaboo-close.patch",
157158
"peekaboo-clipboard-text.patch",
158159
]
159160
.map((name) => join(native, "patches", name));

‎apps/host/src/desktop.ts‎

Lines changed: 13 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -507,7 +507,11 @@ async function act(request: DesktopAction, signal: AbortSignal): Promise<Desktop
507507
}, outcome);
508508
}
509509
const outcome = data.outcome as DesktopOutcome;
510-
if (request.op === "quit" && outcome === "unknown" && data.terminated === false) {
510+
if (
511+
outcome === "unknown"
512+
&& ((request.op === "quit" && data.terminated === false)
513+
|| (request.op === "close" && data.target_receipt))
514+
) {
511515
return await observeManagement(request, data, signal);
512516
}
513517
if (outcome !== "completed" || request.op === "clipboard-write") {
@@ -560,7 +564,7 @@ async function observeManagement(
560564
signal: AbortSignal,
561565
): Promise<DesktopResult> {
562566
const data: Record<string, unknown> = { action };
563-
const outcome = request.op === "quit" && action.outcome === "unknown" ? "unknown" : "completed";
567+
const outcome = action.outcome === "unknown" ? "unknown" : "completed";
564568
const receipt = action.target_receipt as Reply["target_receipt"];
565569
if (
566570
!receipt || receipt.pid !== request.target.pid
@@ -583,8 +587,10 @@ async function observeManagement(
583587
data.application_inventory_warnings = apps.inventory_warnings;
584588
const app = apps.apps.find((app) => app.pid === receipt.pid);
585589
data.application = app || null;
586-
// Quit's native receipt owns termination evidence; later inventory cannot undo or establish it.
587-
if (!app && request.op === "quit") return managementResult(data, action, outcome);
590+
// The native receipt owns close/quit evidence; later inventory cannot undo or establish it.
591+
if (!app && (request.op === "quit" || request.op === "close")) {
592+
return managementResult(data, action, outcome);
593+
}
588594
if (!app) throw new Error("The target application was not returned by the later inventory.");
589595
if (app.process_start_identity_decimal !== receipt.process_start_identity_decimal) {
590596
throw new Error("The application changed process generation after the action.");
@@ -603,7 +609,7 @@ async function observeManagement(
603609
"Later window inventory could not be bound to the original application generation.",
604610
);
605611
}
606-
if (request.op === "activate" || request.op === "quit") {
612+
if (request.op === "activate" || request.op === "quit" || request.op === "close") {
607613
return managementResult(data, action, outcome);
608614
}
609615
if (!windows.windows.some((window) => window.window_id === receipt.window_id)) {
@@ -640,7 +646,7 @@ async function observeManagement(
640646
);
641647
data.message = outcome === "completed"
642648
? "The native action completed. Later inventory or inspection was unavailable; refresh the target before any further action, without repeating the completed action blindly."
643-
: "Termination was not confirmed and later inventory was unavailable. Refresh the target before choosing any further action; do not blindly retry or force quit.";
649+
: "Native completion was not confirmed and later inventory was unavailable. Refresh the target before choosing any further action; do not blindly repeat the request.";
644650
return managementResult(data, action, outcome);
645651
}
646652
}
@@ -684,6 +690,7 @@ export const desktop: Desktop = async (request, context) => {
684690
"drag",
685691
"activate",
686692
"quit",
693+
"close",
687694
"focus",
688695
"minimize",
689696
"restore",

‎docs/architecture.md‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -198,12 +198,15 @@ signing team. Tools expose application and window inventories, observation, elem
198198
clicks, scrolling, atomic drags, replacement of editable field values, text selection/insertion, and keys or shortcuts. Screenshot
199199
points are normalized and mapped through the snapshot's native capture geometry. Control input uses background delivery bound to a
200200
snapshot's exact process, window, and controls; every dispatched action requires a new observation.
201-
Separate activation, quit, focus, minimize, restore, move, and resize tools accept generation-bound inventory targets, with exact
201+
Separate activation, quit, close, focus, minimize, restore, move, and resize tools accept generation-bound inventory targets, with exact
202202
window bounds where applicable. The native service revalidates those receipts before dispatch;
203203
activation and focus explicitly change the foreground desktop. Inspection never activates a target
204204
implicitly. App-only action receipts refresh inventories without inventing a selected window.
205205
Normal quit reports completion only after native termination confirmation; an accepted request that
206206
leaves the app running stays uncertain. It never retries, force-quits, or chooses a dialog response.
207+
Exact-window close selects one supported background Accessibility route before input and never
208+
falls through after dispatch. Confirmed disappearance survives later inventory failure; an accepted
209+
close that remains open is uncertain with unsafe retry and fresh inventory when available.
207210
Minimize verifies native window state and returns fresh inventory without capturing the minimized
208211
window; restoring it remains an explicit action with a refreshed target.
209212
Move and resize use native background Accessibility, verify resulting geometry, and return refreshed

0 commit comments

Comments
 (0)