Exploring Windows Graphics Capture as an alternative capture backend for seamless mode

[samumaiurana08]

Hi,

I have been thinking about a possible improvement for WinApps' seamless mode.

Currently, WinApps relies on FreeRDP RemoteApp to display Windows applications as native Linux windows. This works well, but I was wondering if it would be possible to explore another approach based on the Windows Graphics Capture (WGC) API.

WGC is Microsoft's modern API for capturing individual application windows. Instead of capturing the whole desktop, it can target specific windows and could potentially provide a cleaner and more direct way to integrate Windows applications into the Linux desktop environment.

Some possible advantages:

• Better handling of modern Windows applications that use hardware acceleration.
• Potentially lower latency compared to traditional capture approaches.
• Fewer graphical artifacts in seamless mode.
• A possible path toward a more native-feeling integration.

I understand that integrating this would require significant architectural changes and that WinApps currently relies on FreeRDP for RemoteApp functionality. This is only a suggestion for a possible future backend or experimental mode, not necessarily a replacement for the current approach.

I would like to know if this is technically feasible or if there are limitations in the current architecture that would make this impractical.

Thanks for maintaining this project!

[oskardotglobal]

Hi,

trying alternatives to FreeRDP is a long-term goal for the rewrite. After quickly looking into it: that API doesn‘t handle mouse/keyboard input, does it? While we might be able to get a decent picture to the host, then also having to figure out where the mouse goes is non-trivial. However, it‘s better than manipulating raw pixels i guess

[samumaiurana08]

I was actually thinking along the lines of Sunshine/Moonlight rather than using WGC alone.

The idea would be to use WGC only for efficient window capture, while handling input through a separate channel. Sunshine already solves low-latency video streaming and input forwarding, but it's designed for full desktop streaming.

What I had in mind is something optimized specifically for seamless mode: capturing individual application windows, forwarding mouse and keyboard separately, and presenting them as native Linux windows instead of streaming an entire desktop.

So WGC would only be one piece of the pipeline, not the complete solution.