Direct2D based blur effect in Windows Runtime Apps

Effects such as DropShadowEffect, BlurEffect were removed from Windows Runtime XAML. In order to achieve some goals, I need to write something like these.

Luckily Direct2D provides many useful effects for us, including Gaussian Blur, which is the effect I want.

At first I tried SharpDX, it worked well on Intel platform devices, but not ARM-based devices. To make matters worse, SharpDX‘s performance was not so good as I thought. So I had to write a C++/CX Windows Runtime Component and use it in my own Windows Runtime XAML project.
Here’s the result.

Windows Runtime XAML Render to bitmap sample with blur effect
Windows Runtime XAML render to bitmap sample with blur effect

To use Direct2D, I need to create device resources first. Create the Direct3D 11 API device object, and then get the Direct2D device object.

Note: To convert stream, see here: http://blogs.msdn.com/b/win8devsupport/archive/2013/05/15/how-to-do-data-conversion-in-windows-store-app.aspx

Then receive the bitmap and create WIC object. Finally get things ready and draw, and generate output file.

Note: Set D2D1_GAUSSIANBLUR_PROP_BORDER_MODE to D2D1_BORDER_MODE_HARD, you will get the iOS 7-like blur style.

Here’s the main source code:

D2DEffect.cpp


#include "pch.h"
#include "D2DBlurEffect.h"

using namespace Light::UI::Effects::Direct2D::BlurEffect;
using namespace Platform;
using namespace concurrency;

using namespace Microsoft::WRL;
using namespace Windows::ApplicationModel;
using namespace Windows::System;
using namespace Windows::Foundation;
using namespace Windows::Graphics::Display;
using namespace Windows::Storage;
using namespace Windows::UI::Core;

// Initialize hardware-dependent resources.
void BlurEffectImageProcessor::CreateDeviceResources()
{
// This flag adds support for surfaces with a different color channel ordering
// than the API default. It is required for compatibility with Direct2D.
UINT creationFlags = D3D11_CREATE_DEVICE_BGRA_SUPPORT;

#if defined(_DEBUG)
// If the project is in a debug build, enable debugging via SDK Layers.
creationFlags |= D3D11_CREATE_DEVICE_DEBUG;
#endif

// This array defines the set of DirectX hardware feature levels this app will support.
// Note the ordering should be preserved.
// Don't forget to declare your application's minimum required feature level in its
// description. All applications are assumed to support 9.1 unless otherwise stated.
const D3D_FEATURE_LEVEL featureLevels[] =
{
D3D_FEATURE_LEVEL_11_1,
D3D_FEATURE_LEVEL_11_0,
D3D_FEATURE_LEVEL_10_1,
D3D_FEATURE_LEVEL_10_0,
D3D_FEATURE_LEVEL_9_3,
D3D_FEATURE_LEVEL_9_2,
D3D_FEATURE_LEVEL_9_1,
};

// Create the Direct3D 11 API device object.
DX::ThrowIfFailed(
D3D11CreateDevice(
nullptr, // Specify nullptr to use the default adapter.
D3D_DRIVER_TYPE_HARDWARE,
nullptr,
creationFlags, // Set debug and Direct2D compatibility flags.
featureLevels, // List of feature levels this app can support.
ARRAYSIZE(featureLevels),
D3D11_SDK_VERSION, // Always set this to D3D11_SDK_VERSION for Windows Store apps.
&m_d3dDevice, // Returns the Direct3D device created.
nullptr,
nullptr
)
);

// Get the Direct3D 11.1 API device.
ComPtr dxgiDevice;
DX::ThrowIfFailed(
m_d3dDevice.As(&dxgiDevice)
);

// Create the Direct2D device object and a corresponding context.
DX::ThrowIfFailed(
D2D1CreateDevice(
dxgiDevice.Get(),
nullptr,
&m_d2dDevice
)
);

DX::ThrowIfFailed(
m_d2dDevice->CreateDeviceContext(
D2D1_DEVICE_CONTEXT_OPTIONS_NONE,
&m_d2dContext
)
);
}

///
/// Internal method referred from Bing.
/// Convert IBuffer to IStream.
///

///The buffer to convert. IStream* createIStreamFromIBuffer(Streams::IBuffer ^buffer) {
// convert the IBuffer into an IStream to be used with WIC
IStream *fileContentsStream;
HRESULT res = CreateStreamOnHGlobal(NULL, TRUE, &fileContentsStream);
if (FAILED(res) || !fileContentsStream) {
throw ref new FailureException();
}
Streams::DataReader^ dataReader = Streams::DataReader::FromBuffer(buffer);
// read the data into the stream in chunks of 1MB to preserve memory
while (dataReader->UnconsumedBufferLength > 0) {
UINT chunkSize = min(1024 * 1024, dataReader->UnconsumedBufferLength);
auto data = ref new Platform::Array(chunkSize);
dataReader->ReadBytes(data);
ULONG written;
res = fileContentsStream->Write(data->Data, chunkSize, &written);
if (FAILED(res) || written != chunkSize) {
fileContentsStream->Release();
throw ref new FailureException();
}
}
return fileContentsStream;
}

BlurEffectImageProcessor::BlurEffectImageProcessor()
{
IsInitialized = false;
}

///
/// Render image but not get the final image.
/// REMEMBER call DataInitialize method first.
///

///Indicates the the blur amount. ///Indicates the current display's DPI. IAsyncAction^ BlurEffectImageProcessor::RenderImage(float gaussianBlurStDev, float DPI){
return create_async([this, gaussianBlurStDev,DPI]{
if (!IsInitialized){
throw ref new Platform::Exception(1, "The class has not initialized.");
}

// Render it
UINT imageWidth;
UINT imageHeight;
m_wicFormatConverter->GetSize(&imageWidth, &imageHeight);

// Create a Bitmap Source Effect.
DX::ThrowIfFailed(m_d2dContext->CreateEffect(CLSID_D2D1BitmapSource, &m_bitmapSourceEffect));

// Set the BitmapSource Property to the BitmapSource generated earlier.
DX::ThrowIfFailed(
m_bitmapSourceEffect->SetValue(D2D1_BITMAPSOURCE_PROP_WIC_BITMAP_SOURCE, m_wicFormatConverter.Get())
);

// Create the Gaussian Blur Effect.
DX::ThrowIfFailed(m_d2dContext->CreateEffect(CLSID_D2D1GaussianBlur, &m_gaussianBlurEffect));

// Set the input to recieve the bitmap from the BitmapSourceEffect.
m_gaussianBlurEffect->SetInputEffect(0, m_bitmapSourceEffect.Get());

// Set the blur amount.
DX::ThrowIfFailed(m_gaussianBlurEffect->SetValue(D2D1_GAUSSIANBLUR_PROP_STANDARD_DEVIATION, gaussianBlurStDev));
DX::ThrowIfFailed(m_gaussianBlurEffect->SetValue(D2D1_GAUSSIANBLUR_PROP_BORDER_MODE, D2D1_BORDER_MODE_HARD));

// Begin drawing.
m_d2dContext->BeginDraw();

m_d2dContext->Clear(D2D1::ColorF(D2D1::ColorF::CornflowerBlue));

// Draw the scaled and blurred image.
m_d2dContext->DrawImage(m_gaussianBlurEffect.Get());

// We ignore D2DERR_RECREATE_TARGET here. This error indicates that the device
// is lost. It will be handled during the next call to Present.
HRESULT hr = m_d2dContext->EndDraw();
if (hr != D2DERR_RECREATE_TARGET)
{
DX::ThrowIfFailed(hr);
}

});
}

///
/// Initializes all device resources and the image.
/// You need to call this method before doing other things.
///

IAsyncAction^ BlurEffectImageProcessor::DataInitialize(IRandomAccessStream^ ImageDataStream,float DPI){
// DirectXBase::Initialize(Window, DPI);
return create_async([this,ImageDataStream, DPI]{
// Initialize Devices
CreateDeviceResources();

DX::ThrowIfFailed(CoCreateInstance(
CLSID_WICImagingFactory1,
nullptr,
CLSCTX_INPROC_SERVER,
IID_PPV_ARGS(&m_wicImagingFactory)
)
);

DX::ThrowIfFailed(
CoCreateInstance(
CLSID_WICImagingFactory,
nullptr,
CLSCTX_INPROC_SERVER,
IID_PPV_ARGS(&m_wicImagingFactory2)
)
);

// Now we have the image source and we can decode it.
ImageBuffer = ref new Buffer(ImageDataStream->Size);
auto op = create_task(ImageDataStream->ReadAsync(ImageBuffer, ImageDataStream->Size, InputStreamOptions::None)).then([this,DPI](IBuffer^ ImageBufferData){
DX::ThrowIfFailed(
m_wicImagingFactory2->CreateDecoderFromStream(createIStreamFromIBuffer(ImageBufferData), nullptr, WICDecodeMetadataCacheOnDemand,
&m_wicDecoder)
);

// Get data ready
DX::ThrowIfFailed(
m_wicDecoder->GetFrame(0, &m_wicFrameDecode)
);
DX::ThrowIfFailed(
m_wicImagingFactory2->CreateFormatConverter(&m_wicFormatConverter)
);

DX::ThrowIfFailed(
m_wicFormatConverter->Initialize(
m_wicFrameDecode.Get(),
GUID_WICPixelFormat32bppBGRA,
WICBitmapDitherTypeNone,
nullptr,
0.0f,
WICBitmapPaletteTypeCustom
)
);

// Create output bitmap & get it ready
UINT Width;
UINT Height;
m_wicFrameDecode->GetSize(&Width, &Height);
m_wicImagingFactory2->CreateBitmap(Width, Height, GUID_WICPixelFormat32bppBGRA, WICBitmapCreateCacheOption::WICBitmapCacheOnDemand, &m_wicBitmap);
D2D1_SIZE_U bitmapSize = D2D1::SizeU(Width, Height);
D2D1_PIXEL_FORMAT bitmapPixelFormat = D2D1::PixelFormat(DXGI_FORMAT_B8G8R8A8_UNORM, D2D1_ALPHA_MODE_IGNORE);
D2D1_BITMAP_PROPERTIES1 bitmapProp1 = D2D1::BitmapProperties1(D2D1_BITMAP_OPTIONS_TARGET,bitmapPixelFormat, DPI, DPI);
m_d2dContext->CreateBitmap(
D2D1::SizeU(Width, Height),
nullptr,
Width * 4, // 4 bytes for B8G8R8A8
bitmapProp1,
&m_d2dBitmap1
);

m_d2dContext->SetTarget(m_d2dBitmap1.Get());

IsInitialized = true;

return;
});

op.wait();
});
}

///
/// Get the final image.
/// REMEMBER call DataInitialize method first.
/// You can call this method before calling RenderImage, but you will get the original image.
///

///Indicates the current display's DPI. IAsyncOperation<IRandomAccessStream^>^ BlurEffectImageProcessor::GetImageAsBitmap(float DPI){
return create_async([this,DPI]{
if (!IsInitialized){
throw ref new Platform::Exception(1, "The class has not initialized.");
}
// Render the bitmap use WIC.
ComPtr m_iwicBitmap;
ComPtr m_iwicStream;
ComPtr m_iwicBitmapEncoder;
ComPtr m_iwicBitmapFrameEncode;
ComPtr m_iwicImageEncoder;
WICImageParameters* m_imageparm = new WICImageParameters();
D2D1_PIXEL_FORMAT m_pixel_format = D2D1_PIXEL_FORMAT();
ComPtr m_iStream;
ID2D1Image* m_id2d1image;
UINT height;
UINT width;

// Since we can't create IStream directly in Windows Runtime, we need creating InMemoryRandomAccessStream and convert it
IRandomAccessStream^ data = ref new InMemoryRandomAccessStream();

DX::ThrowIfFailed(
CreateStreamOverRandomAccessStream(data, IID_PPV_ARGS(&m_iStream))
);

// Get size, we need it later
DX::ThrowIfFailed(
m_wicFrameDecode->GetSize(&width, &height)
);

// Create bitmap
DX::ThrowIfFailed(
m_wicImagingFactory2->CreateBitmap(width, height, GUID_WICPixelFormat32bppBGRA, WICBitmapCreateCacheOption::WICBitmapCacheOnDemand, &m_iwicBitmap)
);

// Create WIC Stream
DX::ThrowIfFailed(
m_wicImagingFactory->CreateStream(&m_iwicStream)
);

// Initialize WIC Stream from IStream that we converted
DX::ThrowIfFailed(
m_iwicStream->InitializeFromIStream(m_iStream.Get())
);

// Create encoder
DX::ThrowIfFailed(
m_wicImagingFactory2->CreateEncoder(GUID_ContainerFormatPng, nullptr, &m_iwicBitmapEncoder)
);

// Create image encoder
DX::ThrowIfFailed(
m_wicImagingFactory2->CreateImageEncoder(m_d2dDevice.Get(), &m_iwicImageEncoder)
);

// Initialize
DX::ThrowIfFailed(
m_iwicBitmapEncoder->Initialize(m_iwicStream.Get(), WICBitmapEncoderCacheOption::WICBitmapEncoderNoCache)
);

// Create new frame for the bitmap
DX::ThrowIfFailed(
m_iwicBitmapEncoder->CreateNewFrame(&m_iwicBitmapFrameEncode,nullptr)
);

// Set properties
m_iwicBitmapFrameEncode->Initialize(nullptr);
m_iwicBitmapFrameEncode->SetSize(width, height);
WICPixelFormatGUID format = GUID_WICPixelFormat32bppBGRA;
m_iwicBitmapFrameEncode->SetPixelFormat(&format);
m_d2dContext->GetTarget(&m_id2d1image);
m_imageparm->DpiX = DPI;
m_imageparm->DpiY = DPI;
m_pixel_format.alphaMode = D2D1_ALPHA_MODE_IGNORE;
m_pixel_format.format = DXGI_FORMAT_B8G8R8A8_UNORM;
m_imageparm->PixelFormat = m_pixel_format;
m_imageparm->PixelHeight = height;
m_imageparm->PixelWidth = width;

// Write frmae
DX::ThrowIfFailed(
m_iwicImageEncoder->WriteFrame(m_id2d1image, m_iwicBitmapFrameEncode.Get(), m_imageparm)
);

// Commit
DX::ThrowIfFailed(
m_iwicBitmapFrameEncode->Commit()
);

DX::ThrowIfFailed(
m_iwicBitmapEncoder->Commit()
);

// Now we successfully got the image
// Convert it to stream.
// Reference: MSDN
Windows::Storage::Streams::IRandomAccessStream^ comRAS;
IUnknown* p11 = reinterpret_cast(comRAS);

static const GUID guidIRandomAccessStream =
{ 0x905a0fe1, 0xbc53, 0x11df, { 0x8c, 0x49, 0x00, 0x1e, 0x4f, 0xc6, 0x86, 0xda } };

DX::ThrowIfFailed(
CreateRandomAccessStreamOverStream(m_iwicStream.Get(), BSOS_DEFAULT, guidIRandomAccessStream, (void**)&p11)
);

// Return result
return reinterpret_cast<IRandomAccessStream^>(p11);
});
}


D2DEffects.h

#pragma once

#include "DirectXBase.h"

using namespace Windows::Storage::Streams;
using namespace Windows::Foundation;
using namespace Windows::UI::Core;

namespace Light{
namespace UI{
namespace Effects{
namespace Direct2D{
namespace BlurEffect{
public ref class BlurEffectImageProcessor sealed
{
public:
BlurEffectImageProcessor();
IAsyncAction^ DataInitialize(IRandomAccessStream^ ImageDataStream,float DPI);
IAsyncAction^ RenderImage(float gaussianBlurStDev, float DPI);
IAsyncOperation<IRandomAccessStream^>^ GetImageAsBitmap(float DPI);
void CreateDeviceResources();
private:
Microsoft::WRL::ComPtr m_bitmapSourceEffect;
Microsoft::WRL::ComPtr m_gaussianBlurEffect;

Microsoft::WRL::ComPtr m_wicDecoder;
Microsoft::WRL::ComPtr m_wicFrameDecode;
Microsoft::WRL::ComPtr m_wicFormatConverter;
Microsoft::WRL::ComPtr m_wicImagingFactory2;
Microsoft::WRL::ComPtr m_wicImagingFactory;
Microsoft::WRL::ComPtr m_d2ddevice1;
Microsoft::WRL::ComPtr m_d2ddevice;
Microsoft::WRL::ComPtr m_d3d11device;

// Direct3D device
Microsoft::WRL::ComPtr m_d3dDevice;

// Direct2D objects
Microsoft::WRL::ComPtr m_d2dDevice;
Microsoft::WRL::ComPtr m_d2dContext;
Microsoft::WRL::ComPtr m_d2dBitmap1;
Microsoft::WRL::ComPtr m_wicBitmap;

int m_width;
int m_height;
IBuffer^ ImageBuffer;
bool IsInitialized;
};
}
}
}
}
}


And don’t forget these input file: dxgi, dwrite, d2d1, d3d11, windowscodec, etc.

写在毕业之后 / After graduation

2014.7.13 Update

早上起来看了一下Dashboard,没想到纯粹为了写写个人感受的一段文字能得到同学们那么多的关注。谢谢你们。

我很想向一些同学道歉。我私下里对一些同学还是有意见的(个人因素),还有三年经常管理纪律,在这方面也对一些同学造成了些许伤害。后来我从管理纪律的位置上退下来,一方面有学业压力的原因,另一方面也有怕伤害同学感情的原因。我并不想成为林老师所说的”孤家寡人”。何必呢。

此外还要”张博”的问题。可能就是一个误解。因为自己的圈子里讨论各种东西已经习以为常,然后就潜移默化迁移到了班里。知识栈(这个我编的)太大了也不见得是好事。

手机码字很累,今天还要上课,回来补。

2014.7.12 Origin

毕业之后总是有很多的话想说。总是想写却发现没有时间写,或者对着屏幕除了写代码就没有想法写这些。但无论怎么样还是要写一写,即使断断续续写也要写。

还有WordPress为什么把自然段首行缩进自动干掉了…

首先很抱歉,诸位,张博强父亲在出差,纪念册的源文件我还没有拿到,纸质版不一定很快能做好。但是不管怎么样,会做。其实等待几个月Windows Library for JavaScript更成熟了也可能会更好。私底下我会这份赶工的纪念册的一些小瑕疵有意见,但是怎么说呢,张博强牺牲自己的时间(这还真是牺牲,有些我知道的事情我实在不想说)在这么短的时间里做出这份东西已经很不容易了,特别是中间经历了一次工具链的转换,还要涉及到那么多人的资料收集整理,内容的挑选和录入。(其实张博强很抱歉一开始我应该建议你用Adobe InDesign的)

毕业那天要了很多签名,也签了很多名。今天整理东西才发现我还有几张纸头没有写(也就是要联系方式的),想想真是不好意思。有位同学曾经写过一篇黑我的文章,但那位黑我的同学的纸头我还是写了的。其实私下里我很钦佩这位同学(虽然某些时候我们是竞争对手。

继续说那件事情吧。一开始我不知道那篇文章的存在。直到有一天要读随笔。读完一篇林老师发现了下一篇就直接读了。那篇文章黑我的几个点我基本承认。我承认我的性格总是有点诡异,跟一些同学总是相处不融洽,或是语言表达那么奇怪。后来发生的事情更加意外,完全超出了我的意料,但事后还是去问了当事人的感受。但是可以肯定的是,那天上课她肯定不是那么好过,我也不知道隔壁班的其他同学的反应。但可以肯定那天数学课(我还记得)隔壁班声音好大。后来那天下午总算感觉好一点了。 其实这件事情好早以前就存在了(大概2012.10的时候吧),只是一直不被多数人所知而已。藏了好久,最后终于慢慢被人挖掘出来(默默看ZBQ)。有几次在走廊上和ZBQ一同走,然后经常有路过时ZBQ用诡异的语气说我的名字的情景发生。然后第六次月考我是真心出于看作文的目的(感觉自己书面表达有13分实在不科学),去借了卷子,出于某些原因还是让同学委托完成的,后来不知怎么样就真的泄露了。我也不想管了,最后几十天折腾什么呢。然后就有了好多事情。其实真心地我也很想为我的一些失礼行为道歉。不管怎么说我要感谢你,让我在最后的一段时间里有了精神的支点。

还有同寝室的你(请注意上下文已切换)。初三刚开学的时候,想顺其自然一次,然后就被分到了混合寝室。然后跟你分到了一起。不管怎么说,一年把你的卫生习惯培养得还算满意了。你还是那么不懂事,到了寝室照样和二班的范打打闹闹,玩枕头大战,一次次把你们叫停,然后强迫你们洗漱睡觉。有段时间,大家还在床上做仰卧起坐,乐此不疲。其实现在看来,这些都很有趣,你的不懂事也给我留下的美好的印象。你在中考前还在担心自己发挥失常怎么办。事实证明你发挥正常(超常)了,我也实现了我在寝室里所说的“不让第一志愿生效”的愿望。范,我不知道你情况怎么样了,如果你看到了这段文字请私下里联系我。

(上下文又切换了)然后就是中考。中考卷是有点奇怪,答题卡的颜色也有点奇葩。我不抱任何杂念,纯粹就是经历了五场考试。结果也还满意,去了自己最想去的学校。很不幸你没有发挥好。但也可以用一句话解释:

They say you won’t necessarily succeed if you trybut if you quit, you will necessarily fail.

不要紧,我们三年后US见。还有我相信网络和云的力量,还有我相信Microsoft OneNote这个生产力工具的力量。 然后还有经常同桌吃饭的你、你、和他。我们各自去了不同学校(杭二、学军、杭高、杭二分),同桌吃饭从天文讨论到地理,从化学讨论到物理,从计算机讨论到炸药的情景不会再有了。祝你们在新学校里开心(还有你)。祝你的新App早日上Top Free。保持联系。

然后毕业典礼。感觉华心态真好,自己中考出了这么大意外心情照样还好,还赠送了每个人写了名字的可乐。晚上的Party,邵从来没有完整公开过的健美操也公开展示了,还配的的Bad Apple(一开始我还担心能否跟上)。那天我第一次听到周老师会用这么亲昵的声音叫我。那天留了很多照片,已经被我永久保存。那天给主课老师都送了礼物。那天参与了毕业典礼的布置……

然后我们就那么散了。三年同窗生涯画上句号。

到了高中,特别到了学军的夏令营后,才发现初中的知识是渣渣(这也许跟老师的上课习惯有关)。OneNote里瞬间多了好多还没有消化的知识。哦,还有社团。社团要我接手,好多没用过的东西还要学习使用方法,还要看前辈造的一个个不屑注释的轮子,或是自己用C#C++再造,还要有创意,还要综合各种知识,还要调配时间。我会努力的。 有很多事情想做,迫于精力、金钱等等因素没有如愿完成,更何况现在我要花时间同时兼顾国内课程和出国事宜。但是…

最近还要把Light 2.0的合作开发完成。

最近还要少科院科考。

最近还要日本访问。

最近还要…….

我会努力的。还有你们。

暂时的分别是为了更好的相聚。

— 可能还会想到什么写什么。 —

2014.7.12

Linode大法好 (Linode is just…..awesome!)

呐,整个站迁移到Linode上了。果然访问快了好多好多好多。Linode大法好,早日摆脱共享主机早日获得新生。

需要注意几个事情:

  • Linode的年付优惠我已经找不到了。
  • 日本机房不一定好。我选的是Fremont。
  • Ubuntu Server 14.04 LTS装了php5后需要手动给apache的模块赋执行权限
  • 注意wp-content的权限 (Solved.看下面)
  • 建议2GB以下打开swap
  • (by David Huang)防范DDoS。如果把swap关掉,把mysql的oom_score调到0
  • 不建议搞ftp,要搞iptables做好。如果权限正确更新插件不需要ftp。
  • 把Google Fonts Link换掉。这个搜索一下。不止一个Open Sans,还有Source Code Pro。
  • 2014.7.12 Update 给wp-content赋执行权限,否则会403。

然后这是我的邀请码(就当是资助我吧:https://www.linode.com/?r=88e97b8895dd86f1dc509451440604fc8c313e85