Hack TwinUI to force Windows Store Apps run on low resolution screens

Windows Store Apps on Lumia 640 XL.

Windows 8 and Windows 8.1 has a minimum screen resolution constraint for Windows Store Apps (aka. Metro Apps or whatever). If the screen resolution doesn’t meet requirement, user will see a prompt indicating the resolution is too low for these applications.

However, on certain platforms (like phones and single-board computers), it is not convenient to change resolution. Recently I am trying Windows RT 8.1 on Lumia 640 XL. Qualcomm has the resolution hard-coded in platform configuration, so I was unable to change the resolution. 1280 * 720 is not sufficient for Store Apps.

But there was an exception – the PC settings (aka. Immersive Control Panel) app. It always opens regardless of current resolution settings. So how can I force other applications to launch?

Let’s turn to TwinUI.dll. It’s one of the core components of shell infrastructure. Start IDA Pro, load TwinUI with symbols from Microsoft. Go ahead and search the existence of PC settings app. All Windows Store Apps are associated with a package family identifier. Let’s search it. In this case, it’s windows.immersivecontrolpanel_cw5n1h2txyewy.

Bingo. We found it in some functions.

PC Settings Package Family ID is hardcoded in TwinUI.dll. This function has been patched by me, so it doesn't reflect actual situation you get from official Microsoft binary.
PC Settings Package Family ID is hardcoded in TwinUI.dll. This function has been patched by me, so it doesn’t reflect actual situation you get from official Microsoft binary.

By checking it’s references, we learned that layout checking routine verifies whether it is a desktop application, or PC settings app when resolution doesn’t meet requirements. Either you can patch layout checking routine or PC settings PFN verification routine. I decided to patch the second one, however patching the first is probably a better idea.

On ARMv7-A platform, I simply patched initial register store operation and the branch. Instruction BLX call was replaced with a simple NOP(MOV R0, R0).

Patched function
Patched function

There are two version of the PC settings check routines, so I need to patch both. The other one is similar to this one. Patching the layout verification routine (actually a better idea, as this patch will have some trouble when launch files from desktop) / patching on other architectures should be similar to this one.

Migrate legacy UWP project system to MSBuild-based

When Microsoft decided to adopt MSBuild on .NET Core platform, project.json was not dropped immediately until first toolchain RTM arrives. Dotnet Development on Universal Windows Platform Development leverages .NET Core too, but the depreciation progress is significantly slower than other .NET Core platforms due to historical reasons. UWP uses project.json for package management and MSBuild for builds.

In Visual Studio 2017 April Update, Microsoft finally migrates new UWP projects to full MSBuild-based project system. But our projects, which creates on early 2015, doesn’t get an auto migration as expected. Hence we decided to migrate them manually for additional benefits like better toolchain stability and advanced customization features.

Reminder: Do not attempt to use “dotnet migrate” CLI command, it won’t work for UWP projects.

Migration Prerequisites

  • Notify all your team members. Make sure everyone has Visual Studio 2017 with April update installed.
  • If you have continuous integration environment configured, make sure build agents have NuGet 4.1 or higher installed (3.5 or 4.0 won’t work).
  • Lock VCS during migration to prevent additional incidents. (We’re using TFVC for source management so that it will be easy)

Migration

  • Clean up all projects (including bin and obj directories)
  • Iterate all project directories
  • Find C# project file, open with your favorite editor.
  • Add following property group before project file lists:
<PropertyGroup>
    <RestoreProjectStyle>PackageReference</RestoreProjectStyle>
</PropertyGroup>

Okay, you’ve completed the first step. Then open your project.json file. Migrate all NuGet packages references as the picture below.

Package Reference
Package Reference

Finally, remove project.json and additional files like project.lock.json, *.nuget.targets, *.nuget.props. (Or your will get lots of warning that may lead .NET Native compilation fail)

Do it for every project. Then open Visual Studio, restore NuGet packages for all projects, build to validate and submit changes.

The Windows “Gatekeeper” Internals

"Rickrolling" in Windows SmartScreen

Windows 10 Insider Preview 15046 introduces the Windows-flavor “Gatekeeper“. It is similar to Gatekeeper in macOS, with some minor differences.

First of all, Windows “Gatekeeper” doesn’t block the execution of applications that don’t require installation. I tried to run PuTTY, a popular tool on Windows and it works.

Secondly, Windows “Gatekeeper” is based on Microsoft SmartScreen, which means disabling SmartScreen will turn it off too. Prior to application execution, SmartScreen will send file hash and publisher information(including certificate thumbprint) to Microsoft’s server, then SmartScreen server send back metadata including application reputation. Response is signed with a specific key that will be checked in client side for message integrity.

Unlike macOS, attempt to start application from console(e.g. Command Prompt and PowerShell) will trigger “Gatekeeper”.

Attempt to start application from PowerShell
Attempt to start application from PowerShell

The window is web-based. Although you can’t modify the response directly(no one wants to deal with sha256RSA unless the key leaks), you can attach a debugger to have some fun with it.

"Rickrolling" in Windows SmartScreen
“Rickrolling” in Windows SmartScreen

Microsoft claims that this feature is opt-in for most Windows SKUs (except Windows 10 Cloud AFAIK), and it is not revalent to UMCI (User-mode Code Integrity), which is enforced in Windows 10 Cloud.

Using WASAPI Exclusive mode in Universal Windows Apps (Desktop)

Windows Audio Session API (WASAPI) was first introduced in Windows Vista. It offers advanced audio control and playback features for Windows Apps. Since it mitigates SRC issue in some aspects, WASAPI Exclusive Mode gains its popularity among music lovers. Windows Runtime supports a small subset of WASAPI APIs, including WASAPI Exclusive mode (in Windows 10).

However, here is one thing you should know: Windows 10 Mobile doesn’t support WASAPI Exclusive mode (by design).  As far as I know, only desktop platform is supported yet. Luckily, modern Windows Phone devices can choose the best format for shared mode using input wave’s format, so you don’t have to worry that (at least on Lumia 950 and Lumia 950 XL).

Audio Format

There’s no IMMDevice available in Windows Runtime. Try to create it via CLSID & IID will throw HRESULT Class Not Registered. It will cause some trouble getting all natively supported formats for Exclusive mode. A possible solution is using Properties property in DeviceInformation class in Windows Runtime instead of IMMDevice class & OpenPropertyStore method. Then, query the format using IsFormatSupported in IAudioClient2. Remember that in Exclusive Mode, system won’t return the best-fit wave format in this method, so you have to try all formats and select the best wave format by yourself. In my sample, I specified 44.1kHz / 16Bit / WaveFormat = 0x1, which is supported by my Surface Pro’s audio subsystem.

Initialize Audio Client

Then you can initialize IAudioClient2 in exclusive mode. Simply pass AUDCLNT_SHAREMODE_EXCLUSIVE in, specify Buffer Length and Wave Format. You may get some HRESULTs like AUDCLNT_E_BUFFER_SIZE_NOT_ALIGNED or AUDCLNT_E_BUFFER_SIZE_ERROR. Just check out this page and find solutions.

Perform Playback

Like HW-Offload mode, you don’t have to calculate available frames by yourself In event-based playback mode. The value for padding frames is exactly the same as available frames. Then return audio samples as what you do in shared mode.

Notes

It should supports Windows 8.1 desktop, but I haven’t test it on Windows 8.1 since I don’t have a Windows 8.1 desktop device.

I often noticed weird noise during playback when the system average load is high. I believe the root cause is process priority.

Windows 10 removed Background Audio category in WASAPI headers. In order to implement background playback, a customized out-of-process COM server or Media Foundation extension is required.

I don’t offer a demo program here, because it’s pretty easy to adapt the official sample to WASAPI Exclusive mode. 🙂

Direct2D based blur effect in Windows Runtime Apps

Effects such as DropShadowEffect, BlurEffect were removed from Windows Runtime XAML. In order to achieve some goals, I need to write something like these.

Luckily Direct2D provides many useful effects for us, including Gaussian Blur, which is the effect I want.

At first I tried SharpDX, it worked well on Intel platform devices, but not ARM-based devices. To make matters worse, SharpDX‘s performance was not so good as I thought. So I had to write a C++/CX Windows Runtime Component and use it in my own Windows Runtime XAML project.
Here’s the result.

Windows Runtime XAML Render to bitmap sample with blur effect
Windows Runtime XAML render to bitmap sample with blur effect

To use Direct2D, I need to create device resources first. Create the Direct3D 11 API device object, and then get the Direct2D device object.

Note: To convert stream, see here: http://blogs.msdn.com/b/win8devsupport/archive/2013/05/15/how-to-do-data-conversion-in-windows-store-app.aspx

Then receive the bitmap and create WIC object. Finally get things ready and draw, and generate output file.

Note: Set D2D1_GAUSSIANBLUR_PROP_BORDER_MODE to D2D1_BORDER_MODE_HARD, you will get the iOS 7-like blur style.

Here’s the main source code:

D2DEffect.cpp


#include "pch.h"
#include "D2DBlurEffect.h"

using namespace Light::UI::Effects::Direct2D::BlurEffect;
using namespace Platform;
using namespace concurrency;

using namespace Microsoft::WRL;
using namespace Windows::ApplicationModel;
using namespace Windows::System;
using namespace Windows::Foundation;
using namespace Windows::Graphics::Display;
using namespace Windows::Storage;
using namespace Windows::UI::Core;

// Initialize hardware-dependent resources.
void BlurEffectImageProcessor::CreateDeviceResources()
{
// This flag adds support for surfaces with a different color channel ordering
// than the API default. It is required for compatibility with Direct2D.
UINT creationFlags = D3D11_CREATE_DEVICE_BGRA_SUPPORT;

#if defined(_DEBUG)
// If the project is in a debug build, enable debugging via SDK Layers.
creationFlags |= D3D11_CREATE_DEVICE_DEBUG;
#endif

// This array defines the set of DirectX hardware feature levels this app will support.
// Note the ordering should be preserved.
// Don't forget to declare your application's minimum required feature level in its
// description. All applications are assumed to support 9.1 unless otherwise stated.
const D3D_FEATURE_LEVEL featureLevels[] =
{
D3D_FEATURE_LEVEL_11_1,
D3D_FEATURE_LEVEL_11_0,
D3D_FEATURE_LEVEL_10_1,
D3D_FEATURE_LEVEL_10_0,
D3D_FEATURE_LEVEL_9_3,
D3D_FEATURE_LEVEL_9_2,
D3D_FEATURE_LEVEL_9_1,
};

// Create the Direct3D 11 API device object.
DX::ThrowIfFailed(
D3D11CreateDevice(
nullptr, // Specify nullptr to use the default adapter.
D3D_DRIVER_TYPE_HARDWARE,
nullptr,
creationFlags, // Set debug and Direct2D compatibility flags.
featureLevels, // List of feature levels this app can support.
ARRAYSIZE(featureLevels),
D3D11_SDK_VERSION, // Always set this to D3D11_SDK_VERSION for Windows Store apps.
&m_d3dDevice, // Returns the Direct3D device created.
nullptr,
nullptr
)
);

// Get the Direct3D 11.1 API device.
ComPtr dxgiDevice;
DX::ThrowIfFailed(
m_d3dDevice.As(&dxgiDevice)
);

// Create the Direct2D device object and a corresponding context.
DX::ThrowIfFailed(
D2D1CreateDevice(
dxgiDevice.Get(),
nullptr,
&m_d2dDevice
)
);

DX::ThrowIfFailed(
m_d2dDevice->CreateDeviceContext(
D2D1_DEVICE_CONTEXT_OPTIONS_NONE,
&m_d2dContext
)
);
}

///
/// Internal method referred from Bing.
/// Convert IBuffer to IStream.
///

///The buffer to convert. IStream* createIStreamFromIBuffer(Streams::IBuffer ^buffer) {
// convert the IBuffer into an IStream to be used with WIC
IStream *fileContentsStream;
HRESULT res = CreateStreamOnHGlobal(NULL, TRUE, &fileContentsStream);
if (FAILED(res) || !fileContentsStream) {
throw ref new FailureException();
}
Streams::DataReader^ dataReader = Streams::DataReader::FromBuffer(buffer);
// read the data into the stream in chunks of 1MB to preserve memory
while (dataReader->UnconsumedBufferLength > 0) {
UINT chunkSize = min(1024 * 1024, dataReader->UnconsumedBufferLength);
auto data = ref new Platform::Array(chunkSize);
dataReader->ReadBytes(data);
ULONG written;
res = fileContentsStream->Write(data->Data, chunkSize, &written);
if (FAILED(res) || written != chunkSize) {
fileContentsStream->Release();
throw ref new FailureException();
}
}
return fileContentsStream;
}

BlurEffectImageProcessor::BlurEffectImageProcessor()
{
IsInitialized = false;
}

///
/// Render image but not get the final image.
/// REMEMBER call DataInitialize method first.
///

///Indicates the the blur amount. ///Indicates the current display's DPI. IAsyncAction^ BlurEffectImageProcessor::RenderImage(float gaussianBlurStDev, float DPI){
return create_async([this, gaussianBlurStDev,DPI]{
if (!IsInitialized){
throw ref new Platform::Exception(1, "The class has not initialized.");
}

// Render it
UINT imageWidth;
UINT imageHeight;
m_wicFormatConverter->GetSize(&imageWidth, &imageHeight);

// Create a Bitmap Source Effect.
DX::ThrowIfFailed(m_d2dContext->CreateEffect(CLSID_D2D1BitmapSource, &m_bitmapSourceEffect));

// Set the BitmapSource Property to the BitmapSource generated earlier.
DX::ThrowIfFailed(
m_bitmapSourceEffect->SetValue(D2D1_BITMAPSOURCE_PROP_WIC_BITMAP_SOURCE, m_wicFormatConverter.Get())
);

// Create the Gaussian Blur Effect.
DX::ThrowIfFailed(m_d2dContext->CreateEffect(CLSID_D2D1GaussianBlur, &m_gaussianBlurEffect));

// Set the input to recieve the bitmap from the BitmapSourceEffect.
m_gaussianBlurEffect->SetInputEffect(0, m_bitmapSourceEffect.Get());

// Set the blur amount.
DX::ThrowIfFailed(m_gaussianBlurEffect->SetValue(D2D1_GAUSSIANBLUR_PROP_STANDARD_DEVIATION, gaussianBlurStDev));
DX::ThrowIfFailed(m_gaussianBlurEffect->SetValue(D2D1_GAUSSIANBLUR_PROP_BORDER_MODE, D2D1_BORDER_MODE_HARD));

// Begin drawing.
m_d2dContext->BeginDraw();

m_d2dContext->Clear(D2D1::ColorF(D2D1::ColorF::CornflowerBlue));

// Draw the scaled and blurred image.
m_d2dContext->DrawImage(m_gaussianBlurEffect.Get());

// We ignore D2DERR_RECREATE_TARGET here. This error indicates that the device
// is lost. It will be handled during the next call to Present.
HRESULT hr = m_d2dContext->EndDraw();
if (hr != D2DERR_RECREATE_TARGET)
{
DX::ThrowIfFailed(hr);
}

});
}

///
/// Initializes all device resources and the image.
/// You need to call this method before doing other things.
///

IAsyncAction^ BlurEffectImageProcessor::DataInitialize(IRandomAccessStream^ ImageDataStream,float DPI){
// DirectXBase::Initialize(Window, DPI);
return create_async([this,ImageDataStream, DPI]{
// Initialize Devices
CreateDeviceResources();

DX::ThrowIfFailed(CoCreateInstance(
CLSID_WICImagingFactory1,
nullptr,
CLSCTX_INPROC_SERVER,
IID_PPV_ARGS(&m_wicImagingFactory)
)
);

DX::ThrowIfFailed(
CoCreateInstance(
CLSID_WICImagingFactory,
nullptr,
CLSCTX_INPROC_SERVER,
IID_PPV_ARGS(&m_wicImagingFactory2)
)
);

// Now we have the image source and we can decode it.
ImageBuffer = ref new Buffer(ImageDataStream->Size);
auto op = create_task(ImageDataStream->ReadAsync(ImageBuffer, ImageDataStream->Size, InputStreamOptions::None)).then([this,DPI](IBuffer^ ImageBufferData){
DX::ThrowIfFailed(
m_wicImagingFactory2->CreateDecoderFromStream(createIStreamFromIBuffer(ImageBufferData), nullptr, WICDecodeMetadataCacheOnDemand,
&m_wicDecoder)
);

// Get data ready
DX::ThrowIfFailed(
m_wicDecoder->GetFrame(0, &m_wicFrameDecode)
);
DX::ThrowIfFailed(
m_wicImagingFactory2->CreateFormatConverter(&m_wicFormatConverter)
);

DX::ThrowIfFailed(
m_wicFormatConverter->Initialize(
m_wicFrameDecode.Get(),
GUID_WICPixelFormat32bppBGRA,
WICBitmapDitherTypeNone,
nullptr,
0.0f,
WICBitmapPaletteTypeCustom
)
);

// Create output bitmap & get it ready
UINT Width;
UINT Height;
m_wicFrameDecode->GetSize(&Width, &Height);
m_wicImagingFactory2->CreateBitmap(Width, Height, GUID_WICPixelFormat32bppBGRA, WICBitmapCreateCacheOption::WICBitmapCacheOnDemand, &m_wicBitmap);
D2D1_SIZE_U bitmapSize = D2D1::SizeU(Width, Height);
D2D1_PIXEL_FORMAT bitmapPixelFormat = D2D1::PixelFormat(DXGI_FORMAT_B8G8R8A8_UNORM, D2D1_ALPHA_MODE_IGNORE);
D2D1_BITMAP_PROPERTIES1 bitmapProp1 = D2D1::BitmapProperties1(D2D1_BITMAP_OPTIONS_TARGET,bitmapPixelFormat, DPI, DPI);
m_d2dContext->CreateBitmap(
D2D1::SizeU(Width, Height),
nullptr,
Width * 4, // 4 bytes for B8G8R8A8
bitmapProp1,
&m_d2dBitmap1
);

m_d2dContext->SetTarget(m_d2dBitmap1.Get());

IsInitialized = true;

return;
});

op.wait();
});
}

///
/// Get the final image.
/// REMEMBER call DataInitialize method first.
/// You can call this method before calling RenderImage, but you will get the original image.
///

///Indicates the current display's DPI. IAsyncOperation<IRandomAccessStream^>^ BlurEffectImageProcessor::GetImageAsBitmap(float DPI){
return create_async([this,DPI]{
if (!IsInitialized){
throw ref new Platform::Exception(1, "The class has not initialized.");
}
// Render the bitmap use WIC.
ComPtr m_iwicBitmap;
ComPtr m_iwicStream;
ComPtr m_iwicBitmapEncoder;
ComPtr m_iwicBitmapFrameEncode;
ComPtr m_iwicImageEncoder;
WICImageParameters* m_imageparm = new WICImageParameters();
D2D1_PIXEL_FORMAT m_pixel_format = D2D1_PIXEL_FORMAT();
ComPtr m_iStream;
ID2D1Image* m_id2d1image;
UINT height;
UINT width;

// Since we can't create IStream directly in Windows Runtime, we need creating InMemoryRandomAccessStream and convert it
IRandomAccessStream^ data = ref new InMemoryRandomAccessStream();

DX::ThrowIfFailed(
CreateStreamOverRandomAccessStream(data, IID_PPV_ARGS(&m_iStream))
);

// Get size, we need it later
DX::ThrowIfFailed(
m_wicFrameDecode->GetSize(&width, &height)
);

// Create bitmap
DX::ThrowIfFailed(
m_wicImagingFactory2->CreateBitmap(width, height, GUID_WICPixelFormat32bppBGRA, WICBitmapCreateCacheOption::WICBitmapCacheOnDemand, &m_iwicBitmap)
);

// Create WIC Stream
DX::ThrowIfFailed(
m_wicImagingFactory->CreateStream(&m_iwicStream)
);

// Initialize WIC Stream from IStream that we converted
DX::ThrowIfFailed(
m_iwicStream->InitializeFromIStream(m_iStream.Get())
);

// Create encoder
DX::ThrowIfFailed(
m_wicImagingFactory2->CreateEncoder(GUID_ContainerFormatPng, nullptr, &m_iwicBitmapEncoder)
);

// Create image encoder
DX::ThrowIfFailed(
m_wicImagingFactory2->CreateImageEncoder(m_d2dDevice.Get(), &m_iwicImageEncoder)
);

// Initialize
DX::ThrowIfFailed(
m_iwicBitmapEncoder->Initialize(m_iwicStream.Get(), WICBitmapEncoderCacheOption::WICBitmapEncoderNoCache)
);

// Create new frame for the bitmap
DX::ThrowIfFailed(
m_iwicBitmapEncoder->CreateNewFrame(&m_iwicBitmapFrameEncode,nullptr)
);

// Set properties
m_iwicBitmapFrameEncode->Initialize(nullptr);
m_iwicBitmapFrameEncode->SetSize(width, height);
WICPixelFormatGUID format = GUID_WICPixelFormat32bppBGRA;
m_iwicBitmapFrameEncode->SetPixelFormat(&format);
m_d2dContext->GetTarget(&m_id2d1image);
m_imageparm->DpiX = DPI;
m_imageparm->DpiY = DPI;
m_pixel_format.alphaMode = D2D1_ALPHA_MODE_IGNORE;
m_pixel_format.format = DXGI_FORMAT_B8G8R8A8_UNORM;
m_imageparm->PixelFormat = m_pixel_format;
m_imageparm->PixelHeight = height;
m_imageparm->PixelWidth = width;

// Write frmae
DX::ThrowIfFailed(
m_iwicImageEncoder->WriteFrame(m_id2d1image, m_iwicBitmapFrameEncode.Get(), m_imageparm)
);

// Commit
DX::ThrowIfFailed(
m_iwicBitmapFrameEncode->Commit()
);

DX::ThrowIfFailed(
m_iwicBitmapEncoder->Commit()
);

// Now we successfully got the image
// Convert it to stream.
// Reference: MSDN
Windows::Storage::Streams::IRandomAccessStream^ comRAS;
IUnknown* p11 = reinterpret_cast(comRAS);

static const GUID guidIRandomAccessStream =
{ 0x905a0fe1, 0xbc53, 0x11df, { 0x8c, 0x49, 0x00, 0x1e, 0x4f, 0xc6, 0x86, 0xda } };

DX::ThrowIfFailed(
CreateRandomAccessStreamOverStream(m_iwicStream.Get(), BSOS_DEFAULT, guidIRandomAccessStream, (void**)&p11)
);

// Return result
return reinterpret_cast<IRandomAccessStream^>(p11);
});
}


D2DEffects.h

#pragma once

#include "DirectXBase.h"

using namespace Windows::Storage::Streams;
using namespace Windows::Foundation;
using namespace Windows::UI::Core;

namespace Light{
namespace UI{
namespace Effects{
namespace Direct2D{
namespace BlurEffect{
public ref class BlurEffectImageProcessor sealed
{
public:
BlurEffectImageProcessor();
IAsyncAction^ DataInitialize(IRandomAccessStream^ ImageDataStream,float DPI);
IAsyncAction^ RenderImage(float gaussianBlurStDev, float DPI);
IAsyncOperation<IRandomAccessStream^>^ GetImageAsBitmap(float DPI);
void CreateDeviceResources();
private:
Microsoft::WRL::ComPtr m_bitmapSourceEffect;
Microsoft::WRL::ComPtr m_gaussianBlurEffect;

Microsoft::WRL::ComPtr m_wicDecoder;
Microsoft::WRL::ComPtr m_wicFrameDecode;
Microsoft::WRL::ComPtr m_wicFormatConverter;
Microsoft::WRL::ComPtr m_wicImagingFactory2;
Microsoft::WRL::ComPtr m_wicImagingFactory;
Microsoft::WRL::ComPtr m_d2ddevice1;
Microsoft::WRL::ComPtr m_d2ddevice;
Microsoft::WRL::ComPtr m_d3d11device;

// Direct3D device
Microsoft::WRL::ComPtr m_d3dDevice;

// Direct2D objects
Microsoft::WRL::ComPtr m_d2dDevice;
Microsoft::WRL::ComPtr m_d2dContext;
Microsoft::WRL::ComPtr m_d2dBitmap1;
Microsoft::WRL::ComPtr m_wicBitmap;

int m_width;
int m_height;
IBuffer^ ImageBuffer;
bool IsInitialized;
};
}
}
}
}
}


And don’t forget these input file: dxgi, dwrite, d2d1, d3d11, windowscodec, etc.