Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions dist/build.js
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ const patches = [
"peekaboo-lock-holder.patch",
"peekaboo-focus-activation.patch",
"peekaboo-preparation-message.patch",
"peekaboo-sck-quarantine.patch",
].map((name) => join(native, "patches", name));
function spawn(command, env) {
return spawnSync(command[0], command.slice(1), { encoding: "utf8", env: env ?? process.env });
Expand Down
2 changes: 1 addition & 1 deletion dist/client.js
Original file line number Diff line number Diff line change
Expand Up @@ -372,7 +372,7 @@ async function availability(native, request, signal) {
if (signal.aborted)
return { error: "The desktop call ended before availability could be read." };
const deadline = new AbortController();
const timer = setTimeout(() => deadline.abort(), 2500);
const timer = setTimeout(() => deadline.abort(), 3500);
try {
const context = AbortSignal.any([signal, deadline.signal]);
const results = await Promise.allSettled([
Expand Down
2 changes: 2 additions & 0 deletions docs/tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ Targets are exact: an app target binds a process generation, and a window target
| `desktop_dialog` | Read the open dialog or sheet in an app or window: its buttons, text fields and labels |
| `desktop_clipboard_read` | Read the clipboard as text, an image, or file references |

Captures go through ScreenCaptureKit, which every Peekaboo process for the user shares one call at a time. When an earlier capture timed out and hasn't returned, which usually means another capture on the Mac is busy, a new capture waits up to 2 seconds for it. If it still hasn't returned, the capture is refused with a message saying the problem is temporary; wait a few seconds and inspect again. There is no fallback capture, because every other capture path would also use ScreenCaptureKit while the abandoned call still runs. `desktop_apps` has 2.5 seconds to read the app inventory and is refused, not cut short, when the Mac is too busy to finish in time.

## Apps and windows

| Tool | Does |
Expand Down
21 changes: 21 additions & 0 deletions native/patches/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,27 @@ outcome stays indeterminate, since the activation or click may have reached the
no keys or paste were sent. The caller sends keys only after preparation returns. The existing preparation
test checks the new wording.

`peekaboo-sck-quarantine.patch` makes a quarantined ScreenCaptureKit gate wait briefly and then
explain itself. When an SCK call times out, Peekaboo abandons it and quarantines the gate (and keeps
its cross-process lock) until that call's late callback arrives. A new capture now waits up to 2 s
for that callback before refusing, outside its own budget. The refusal says the quarantine is
temporary, is usually caused by another busy capture on the Mac, and clears by itself; it asks the
caller to wait a few seconds and retry. A timed-out wait for another process's capture says the same.
The private `SCShareableContent.fetchWindowForWindowID` lookup now has 3 s instead of 1 s: under
ordinary load its 1 s budget timed out and quarantined every later capture. The application inventory
budget grows from 1 s to 2.5 s, since a healthy inventory took 0.95 s; upstream deliberately refuses
an inventory that misses its deadline instead of returning it partial, so this patch keeps that.

There is deliberately no fallback around a quarantine. Peekaboo's "CGWindowList" window engine
captures through the private SCK lookup or `/usr/sbin/screencapture -l`, and `screencapture` itself
uses ScreenCaptureKit in its own process. `CGWindowListCreateImage` is obsoleted. Every remaining
capture path would therefore start ScreenCaptureKit beside the abandoned call, which upstream rules
out (`allowsSystemFallback`, "Private SCK fallback never crosses an abandoned operation"). Don't add
a fallback here; wait, refuse, and let the caller retry. The SCK gate lock stays per-user in
`$TMPDIR`, shared with every Peekaboo process; keying it to the host app would let separate
apps overlap ScreenCaptureKit calls again, which is what it prevents. `ScreenCaptureKitQuarantineGraceTests.swift` covers the
grace period and the message.

`desktop-tools-build` resolves only `Package.resolved` versions, checks the pin and checkout revision,
and assembles the ordered patch stack in a private Git index. A build compares the checkout
with each complete ordered prefix, since later patches can change earlier patch contexts.
Expand Down
232 changes: 232 additions & 0 deletions native/patches/peekaboo-sck-quarantine.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,232 @@
diff --git a/Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/Capture/LegacyScreenCaptureOperator+PrivateScreenCaptureKit.swift b/Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/Capture/LegacyScreenCaptureOperator+PrivateScreenCaptureKit.swift
index cb23957cf..928e82c4e 100644
--- a/Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/Capture/LegacyScreenCaptureOperator+PrivateScreenCaptureKit.swift
+++ b/Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/Capture/LegacyScreenCaptureOperator+PrivateScreenCaptureKit.swift
@@ -126,8 +126,9 @@ extension LegacyScreenCaptureOperator {
// WindowServer ID through `SCShareableContent` before building a desktop-independent window filter.
// Public `SCShareableContent.windows` enumeration can miss windows that this lookup still captures.
// If Apple removes this selector, callers fall back to `/usr/sbin/screencapture -l` and then public SCK.
+ // A one-second budget timed out under ordinary load and quarantined every later capture.
return try await ScreenCaptureKitCaptureGate.runOwnedOperation(
- seconds: 1.0,
+ seconds: 3.0,
operationName: "SCShareableContent.fetchWindowForWindowID")
{
try await ScreenCaptureKitCallbackBridge<SCWindow>.wait { finish in
diff --git a/Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/Capture/ScreenCaptureKitCaptureGate.swift b/Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/Capture/ScreenCaptureKitCaptureGate.swift
index c2c26d907..d0c0afd9e 100644
--- a/Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/Capture/ScreenCaptureKitCaptureGate.swift
+++ b/Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/Capture/ScreenCaptureKitCaptureGate.swift
@@ -213,8 +213,12 @@ enum ScreenCaptureKitCallbackBridge<Value: Sendable> {

/// Owns the complete lifetime of one ScreenCaptureKit call, including an SCK call that ignores caller cancellation.
/// A timed-out call quarantines this coordinator until that exact call finishes, preventing orphan accumulation.
+/// New calls wait a short grace period for that late completion before refusing; nothing falls back around a
+/// quarantine, because every capture path left would start ScreenCaptureKit beside the abandoned call.
@MainActor
final class ScreenCaptureKitOperationCoordinator {
+ static let defaultQuarantineGraceSeconds: TimeInterval = 2
+
private enum LeasePhase: Equatable {
case acquiring
case running
@@ -235,10 +239,12 @@ final class ScreenCaptureKitOperationCoordinator {
}

private let lockFilePath: String?
+ private let quarantineGraceSeconds: TimeInterval
private var activeLease: Lease?

- init(lockFilePath: String?) {
+ init(lockFilePath: String?, quarantineGraceSeconds: TimeInterval = defaultQuarantineGraceSeconds) {
self.lockFilePath = lockFilePath
+ self.quarantineGraceSeconds = quarantineGraceSeconds
}

func run<T: Sendable>(
@@ -246,6 +252,8 @@ final class ScreenCaptureKitOperationCoordinator {
operationName: String,
operation: @escaping @MainActor @Sendable () async throws -> T) async throws -> T
{
+ // The grace period is not part of this call's own budget, so a late completion leaves it a full window.
+ try await self.waitForQuarantineToClear()
let deadline = ContinuousClock.now.advanced(by: .seconds(max(seconds, 0)))
let lease = try await self.reserveLease(
operationName: operationName,
@@ -285,8 +293,7 @@ final class ScreenCaptureKitOperationCoordinator {
{
while let activeLease = self.activeLease {
if activeLease.phase == .quarantined {
- throw OperationError.captureFailed(
- reason: "ScreenCaptureKit is quarantined after an abandoned \(activeLease.operationName) call")
+ throw Self.quarantineError(abandonedOperation: activeLease.operationName)
}

try Task.checkCancellation()
@@ -307,6 +314,24 @@ final class ScreenCaptureKitOperationCoordinator {
return lease
}

+ private func waitForQuarantineToClear() async throws {
+ let graceDeadline = ContinuousClock.now.advanced(by: .seconds(max(self.quarantineGraceSeconds, 0)))
+ while let activeLease = self.activeLease, activeLease.phase == .quarantined {
+ try Task.checkCancellation()
+ guard ContinuousClock.now < graceDeadline else {
+ throw Self.quarantineError(abandonedOperation: activeLease.operationName)
+ }
+ try await Self.sleepBeforeRetry(deadline: graceDeadline)
+ }
+ }
+
+ nonisolated static func quarantineError(abandonedOperation: String) -> PeekabooError {
+ OperationError.captureFailed(
+ reason: "ScreenCaptureKit is quarantined for now: an earlier \(abandonedOperation) call in this " +
+ "process timed out and has not returned, usually because another capture on this Mac is busy. " +
+ "This is temporary and clears when that call returns; wait a few seconds and retry")
+ }
+
private func acquireCrossProcessLock(
for lease: Lease,
deadline: ContinuousClock.Instant,
@@ -441,10 +466,13 @@ final class ScreenCaptureKitOperationCoordinator {
waitingForCrossProcessGate: Bool = false) throws
{
guard ContinuousClock.now < deadline else {
- let operation = waitingForCrossProcessGate
- ? "\(operationName) waiting for the ScreenCaptureKit gate; another process may be quarantined"
- : operationName
- throw OperationError.timeout(operation: operation, duration: timeoutSeconds)
+ guard waitingForCrossProcessGate else {
+ throw OperationError.timeout(operation: operationName, duration: timeoutSeconds)
+ }
+ throw PeekabooError.timeout(
+ "\(operationName) waited \(String(format: "%.1f", timeoutSeconds))s for another process's " +
+ "ScreenCaptureKit capture to finish; another process may be quarantined. This is temporary; " +
+ "wait a few seconds and retry")
}
}

diff --git a/Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/System/ApplicationService+Discovery.swift b/Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/System/ApplicationService+Discovery.swift
index 7399df17e..e0b0cdbc9 100644
--- a/Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/System/ApplicationService+Discovery.swift
+++ b/Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/System/ApplicationService+Discovery.swift
@@ -405,7 +405,7 @@ extension ApplicationService {
throw PeekabooError.timeout(
"Application inventory did not complete within its " +
"\(Self.formatInventoryTimeout(self.applicationInventoryOverallTimeout))s observation budget; " +
- "native inventory work may still be in progress")
+ "native inventory work may still be in progress. This usually means the Mac is busy; retry shortly")
}
}

diff --git a/Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/System/ApplicationService.swift b/Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/System/ApplicationService.swift
index 7076739f3..61b7985a9 100644
--- a/Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/System/ApplicationService.swift
+++ b/Core/PeekabooAutomationKit/Sources/PeekabooAutomationKit/Services/System/ApplicationService.swift
@@ -196,7 +196,8 @@ public final class ApplicationService: ApplicationServiceProtocol, ApplicationMu
/// Metadata has eight concurrent waits per inventory, each limited by the remaining budget. The separate
/// process-wide metadata pool retains at most eight native operations through autorelease cleanup.
static let applicationMetadataTimeout: TimeInterval = 0.25
- static let applicationInventoryOverallTimeout: TimeInterval = 1
+ /// One second left no headroom: a healthy inventory took 0.95s, and a loaded Mac refused it outright.
+ static let applicationInventoryOverallTimeout: TimeInterval = 2.5
static let maximumConcurrentApplicationMetadataReads = 8

public convenience init(
diff --git a/Core/PeekabooAutomationKit/Tests/PeekabooAutomationKitTests/ApplicationInventoryTimeoutTests.swift b/Core/PeekabooAutomationKit/Tests/PeekabooAutomationKitTests/ApplicationInventoryTimeoutTests.swift
index 47b6283d5..04bb609e4 100644
--- a/Core/PeekabooAutomationKit/Tests/PeekabooAutomationKitTests/ApplicationInventoryTimeoutTests.swift
+++ b/Core/PeekabooAutomationKit/Tests/PeekabooAutomationKitTests/ApplicationInventoryTimeoutTests.swift
@@ -155,11 +155,12 @@ struct ApplicationInventoryTimeoutTests {
@MainActor
func `PID discovery consumes the overall budget before empty inventory can be complete`() async throws {
let clock = AutomationTestLockedValue(ContinuousClock.now)
+ let pastBudget = ApplicationService.applicationInventoryOverallTimeout + 1
let service = ApplicationService(
applicationOpenHandler: { _, _, _ in throw ApplicationInventoryFixtureError.unused },
frontmostProcessIdentifierProvider: { nil },
runningApplicationProcessIdentifiersProvider: {
- clock.withValue { $0 = $0.advanced(by: .seconds(2)) }
+ clock.withValue { $0 = $0.advanced(by: .seconds(pastBudget)) }
return []
},
applicationWindowCatalogProvider: { [] },
diff --git a/Core/PeekabooAutomationKit/Tests/PeekabooAutomationKitTests/ScreenCaptureKitQuarantineGraceTests.swift b/Core/PeekabooAutomationKit/Tests/PeekabooAutomationKitTests/ScreenCaptureKitQuarantineGraceTests.swift
new file mode 100644
index 000000000..62995c5c0
--- /dev/null
+++ b/Core/PeekabooAutomationKit/Tests/PeekabooAutomationKitTests/ScreenCaptureKitQuarantineGraceTests.swift
@@ -0,0 +1,73 @@
+import Foundation
+import PeekabooAutomationKitTestSupport
+import PeekabooFoundation
+import Testing
+@testable import PeekabooAutomationKit
+
+@MainActor
+struct ScreenCaptureKitQuarantineGraceTests {
+ @Test
+ func `a late completion inside the grace period admits the waiting call`() async throws {
+ let coordinator = ScreenCaptureKitOperationCoordinator(lockFilePath: nil, quarantineGraceSeconds: 5)
+ let release = try await self.quarantine(coordinator)
+
+ let waitingCall = Task { @MainActor in
+ try await coordinator.run(seconds: 0.5, operationName: "waiting capture") { 2 }
+ }
+ try await Task.sleep(for: .milliseconds(100))
+ await release.open()
+
+ #expect(try await waitingCall.value == 2)
+ #expect(!coordinator.isQuarantined)
+ }
+
+ @Test
+ func `an expired grace period refuses with a temporary retry message`() async throws {
+ let coordinator = ScreenCaptureKitOperationCoordinator(lockFilePath: nil, quarantineGraceSeconds: 0.1)
+ let release = try await self.quarantine(coordinator)
+ defer { Task { await release.open() } }
+
+ var admitted = false
+ let start = ContinuousClock.now
+ do {
+ _ = try await coordinator.run(seconds: 0.5, operationName: "refused capture") {
+ admitted = true
+ return 2
+ }
+ Issue.record("A quarantined capture gate admitted another operation")
+ } catch let error as PeekabooError {
+ #expect(error.userMessage.contains("an earlier blocked capture call"))
+ #expect(error.userMessage.contains("temporary"))
+ #expect(error.userMessage.contains("retry"))
+ }
+ #expect(!admitted)
+ #expect(start.duration(to: .now) >= .milliseconds(100))
+ #expect(start.duration(to: .now) < .seconds(2))
+ }
+
+ @Test
+ func `the quarantine refusal blames another capture`() {
+ let error = ScreenCaptureKitOperationCoordinator.quarantineError(abandonedOperation: "fixture")
+ #expect(error.userMessage.contains("another capture on this Mac"))
+ }
+
+ /// Times out one blocked call so the coordinator stays quarantined until the returned latch opens.
+ private func quarantine(_ coordinator: ScreenCaptureKitOperationCoordinator) async throws -> AsyncTestLatch {
+ let started = AsyncTestLatch()
+ let release = AsyncTestLatch()
+ let blocked = Task { @MainActor in
+ try await coordinator.run(seconds: 0.05, operationName: "blocked capture") {
+ await started.open()
+ await release.wait()
+ return 1
+ }
+ }
+ await started.wait()
+ _ = await blocked.result
+ for _ in 0..<100 where !coordinator.isQuarantined {
+ try await Task.sleep(for: .milliseconds(10))
+ }
+ #expect(coordinator.isQuarantined)
+ return release
+ }
+}
1 change: 1 addition & 0 deletions src/build.ts
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ const patches = [
"peekaboo-lock-holder.patch",
"peekaboo-focus-activation.patch",
"peekaboo-preparation-message.patch",
"peekaboo-sck-quarantine.patch",
].map((name) => join(native, "patches", name));

function spawn(command: string[], env?: NodeJS.ProcessEnv) {
Expand Down
2 changes: 1 addition & 1 deletion src/client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -465,7 +465,7 @@ async function availability(
): Promise<Record<string, unknown>> {
if (signal.aborted) return { error: "The desktop call ended before availability could be read." };
const deadline = new AbortController();
const timer = setTimeout(() => deadline.abort(), 2500);
const timer = setTimeout(() => deadline.abort(), 3500);
try {
const context = AbortSignal.any([signal, deadline.signal]);
const results = await Promise.allSettled([
Expand Down
Loading