Blog @ Ben | imbushuo

HWND Hosting on the XAML Composition Visual Tree

Posted at 2022-05-24 Comments Windows IDA XAML WinRT Win32 DirectComposition Reverse Engineering 

During the development of an undisclosed personal project, I encountered a scenario that requires HWND hosting on the XAML surface. While a few alternative approaches are available, I decided to proceed with the Microsoft.UI.Composition route for the best performance. However, this is not an officially supported scenario. Some discussion has existed since 2020, but it remains not formally supported at the time of writing. However, the underlying infrastructure for HWND hosting already exists; just a little bit of reverse engineering effort is required.

Some Alternatives, and Why not

• XAML Island hosting: A bunch issues with WinUI 3.
• Child Window Nesting: This issue declared it was the solution; However, it’s not possible because XAML composition visual fills the entire surface and make it invisible.
• Platform IDirectComposition: Visual Tree issue with Microsoft.UI.Composition, see next section.
• Manual HWND bitmap capture and present in XAML: Surely it works (reinventing poor man’s Swapchain), but involves multi passes of copies between CPU and GPU, not resource friendly.
• WinRT Capture / Desktop Duplication API and XAML Swapchain: better than the solution above (no cross device boundary copy), but still have more resource overhead than DirectComposition.

Review CreateSurfaceFromHwnd

In the platform DirectComposition implementation, IDCompositionDevice::CreateSurfaceFromHwnd exists to accommodate HWND hosting:

HRESULT CreateSurfaceFromHwnd(
    HWND     hwnd,
    IUnknown **surface
);

The usage is straightforward - pass in an HWND owned by the process, get an IUnknown back, call IDCompositionVisual::SetContent to put it into a visual, and commit the visual tree. While the interface and method exist in the undocked DirectComposition (Microsoft.UI.Composition), the required infrastructure supporting HWND Bitmap hosting is stripped. Indeed it’s possible to create a separate IDCompositionDevice and IDCompositionVisual from the platform DirectComposition and place them over the top of the XAML surface. However, since the dedicated DComp device has no visibility into XAML visual tree, it makes XAML visuals behind the HWND hosting visually invisible.

SystemVisualProxyVisualPrivate

Enter SystemVisualProxyVisualPrivate. Microsoft.UI.Composition.Private.SystemVisualProxyVisualPrivate is a special proxy visual implemented in the undocked DirectComposition that bridges visual to the platform DirectComposition. It looks like a normal IVisual at Microsoft.UI.Composition, and it’s a composition target at Windows.UI.Composition. Since the platform composition has proper CreateSurfaceFromHwnd, it’s possible to use SystemVisualProxyVisualPrivate bridging platform HWND visual into WAS XAML composition visual tree.

To get started with SystemVisualProxyVisualPrivate, a few undocumented classes need to be activated manually. Below are their interfaces:

[WindowsRuntimeType("Microsoft.UI")]
[System.Runtime.InteropServices.Guid("6efeef10-e0c5-5997-bcb7-c1644f1cab81")]
[ContractVersion(typeof(WindowsAppSDKContract), 65536u)]
// WinRT IInspectable
internal interface ISystemVisualProxyVisualPrivateStatics
{
    SystemVisualProxyVisualPrivate Create(Compositor compositor);
}

[WindowsRuntimeType("Microsoft.UI")]
[System.Runtime.InteropServices.Guid("B2CFCBC2-7133-4EF8-A686-DB7FD4D536B4")]
[ContractVersion(typeof(WindowsAppSDKContract), 65536u)]
// This is IUnknown, I mistakenly handled it as IInspectable initially and spent
// quite some time in WinDbg wondering why GetHandle breakpoint is not hit
internal interface ISystemVisualProxyVisualPrivateInterop
{
    IntPtr GetHandle();
}

Manually acquire the WinRT activation factory of Microsoft.UI.Composition.Private.SystemVisualProxyVisualPrivate in dcompi.dll, QI it to ISystemVisualProxyVisualPrivateStatics, and then create a SystemVisualProxyVisualPrivate class instance with the current Microsoft.UI.Composition.Compositor from ElementCompositionPreview. From there, QI SystemVisualProxyVisualPrivate instance to ISystemVisualProxyVisualPrivateInterop and get the shared visual target handle.

Now proceed to the platform compositor side: follow the procedure outlined in this blog post to get a Windows.UI.Composition.Compositor and IDCompositionDesktopDevice concurrently. From platform compositor, QI to get IPartner (9CBD9312-070d-4588-9bf3-bbf528cf3e84) and call OpenShardTargetFromHandle to retrieve IVisualTargetPartner instance.

Using the IDCompositionDesktopDevice to perform the required operations getting surface and visual from HWND. QI it to retrieve Windows.UI.Visual instance. Finally, set the wrapped visual as the root visual from the IVisualTargetPartner we retrieved earlier (and also remember to set visual size and scaling), commit changes from both the platform and undocked Compositor.

The full demo code is available at here.

Known Caveats

While this solution works for my case, it might not be a generic solution for everyone. The following are known limitations:

• Visual Tree Air Space issue. Similar to HWND hosting in WPF, anything under the proxy visual is invisible. Certain elements above the visual might have rendering issues (bug?)
• The HWND doesn’t accept input from this projection. Manual hit testing and input message forwarding is required.
• The proxy visual has no awareness of tab stop and other accessibility features. Tab and other accessibility features need to be explicitly handled.

Acknowledgements

Thanks for ADeltaX for getting some IIDs for me, and Rafael Rivera for the idea in the screenshot.