Skip to content

Mitigate black-frame capture for Simulator external display scenes #1

Description

@atjsh

Summary

Capturing the iOS Simulator external display currently produces a black frame even when the routed external-display flow itself is working.

This blocks automated screenshot/evidence capture for external-monitor scenarios, including the live Docker SSH + neofetch flow.

Repro

Environment used locally:

  • Repo: atjsh/Glassdeck
  • Branch: main
  • Simulator: iPhone 17 (B703B3A6-69A3-4B2C-815B-45BC91E16265)
  • External display mode: 1920×1080 (1080p)
  • Live target: Docker SSH fixture on 192.168.0.45:22222

Confirmed live route:

  • A focused UI probe that launched a live Docker session, waited for terminal output, opened Display Routing, and routed the active session to the external display passed locally.
  • The routed session row was exposed as glassdeck@192.168.0.45 in the External Display sheet.
  • Result bundle: /.build/glassdeck-build/results/test/20260324-065449-ui.xcresult

Confirmed capture failures

After the route succeeds, the external scene still captures as black through multiple paths:

  1. Device-side external screenshot
xcrun simctl io B703B3A6-69A3-4B2C-815B-45BC91E16265 screenshot --display external /tmp/external.png

Result: PNG written successfully, but image content is black.

Image
  1. Host-side window capture of the Simulator external display window
screencapture -x -R<external-window-rect> /tmp/external-window.png

Result: captured window content is black.

  1. External display video capture also did not produce a useful visible frame during local verification.

Additional note

A temporary local experiment also forced a plain SwiftUI summary overlay into ExternalTerminalView, gated behind UI-test-only env. That overlay was still not visible in the captured external display output.

That suggests this may be broader than the Ghostty-backed terminal surface alone and may involve how the Simulator external scene is presented/captured.

Impact

  • Blocks automated screenshots for external-monitor documentation and acceptance evidence.
  • Makes external-display regressions harder to prove in CI or reproducible local capture flows.
  • Leaves us without a supported way to produce visual artifacts for routed live sessions.

Goal

Provide a supported, repeatable way to capture non-black external-display output for Simulator external scenes.

Possible directions

  • Investigate whether Simulator external-display capture drops the rendered scene after route activation.
  • Verify whether the problem is specific to Metal/Ghostty-backed surfaces or affects the entire external UIWindowScene capture path.
  • Add a deterministic app-supported capture/export seam for external-display content in UI-test mode if Simulator capture is unreliable.
  • Add a black-frame detection check so capture failures surface immediately with diagnostics instead of silently producing unusable artifacts.
  • Document the supported capture path once confirmed.

Definition of done

  • Reproduce the issue with a committed, deterministic external-display scenario.
  • Produce a non-black external-display artifact through at least one supported automated capture path.
  • Keep the mitigation general enough to cover external-monitor evidence capture, not just the Docker SSH case.
  • Document the chosen capture path and its limits.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    Status
    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions