Give TianoCore/EDK2 on AArch64 a hand in 2018

Windows 10 PE runs on a Spandragon 410 processor.

Also posted here: Bringing up Windows 10 AArch64 on a $50 Single Board Computer

Windows on ARM is not a new topic. There are some guys attempted to bring up Windows RT and Windows 10 on Qemu (ARM/AArch64 target). It even runs on Raspberry Pi 3. Obviously it is not a Snapdragon 835-only thing. We can give it a hand on our own Single Board Computers.

This article covers some important details in Dragonboard 410c SBC’s aa64 UEFI implementation.

Contents

    • Windows Boot Requirements
    • Bootstrapping your own EDK2/TianoCore UEFI
    • Memory Allocation / Memory Management Unit
    • UEFI Flash Definition
    • First-stage Bootloader (Little Kernel)
    • Persistent NVRAM Support
    • A “Working” RTC
    • Multi-processor startup (PSCI)

Windows Boot Requirements (AArch64)

  • AArch64 architecture processor. It seems that AArch64 cryptography extension is required too (Raspberry Pi 3 randomly throws UNSUPPORTED_PROCESSOR bugcheck, rs4 fixed the issue). The bugcheck is raised in Errata Check (a hardcoded ID check).
  • For multi-processor systems, either Microsoft ARM Multi-processor Parking Protocol or ARM PSCI interface shall be implemented. All current Windows 10 IoT ARM32 platforms implement former one.
  • A working interrupt controller. Most AArch64 SoC cores include ARM GIC, so there’s little work to do here. The only exception I know is BCM2837. Windows has inbox Broadcom interrupt controller support (for the sake of Raspberry Pi). But if your SoC has additional third party interrupt controller, you need to supply your own HAL extension library. There is few documentation for this available though…
  • A working processor timer. If not, supply your own HAL extension library.
  • Complete ACPI 5.1/6.0 and UEFI 2.3+ implementation. Do not try to use Das U-Boot’s EFI implementation; it’s broken.

These requirements are fairly similar to ARM SBBR certification requirements. If your SBC has a working EDK2/TianoCore UEFI, then you are probably good to go. Bootstrapping your own EDK2 is pretty easy too.

Bootstrapping your own EDK2/TianoCore

The board I used (DragonBoard 410c) doesn’t have a known EDK2/TianoCore implementation. So I have to build my own. This repository for Raspberry Pi 3 is a good start point and reference for you.

You need to do these things in UEFI:

  • Initialize serial output (for debugging) and Memory Management Unit (MMU). Refer to your platform datasheet for device memory address allocation.
  • Retrieve required information from pre-UEFI environment and build Hand-off Blocks (HOB) for DXE phase
  • Initialize processor (exception vector, etc.) in DXE phase.
  • Initialize required peripherals (GPIO, GIC, eMMC, USB, RTC, Display…) in DXE phase.
  • Initialize UEFI services (variable services) in DXE phase.
  • Jump to BDS phase, start Windows Boot Manager or something else.

Memory Allocation / Memory Management Unit

Memory allocation is a platform-specific thing. Check your platform HRD to get some idea about MMU and memory allocation. For Snapdragon 410, check out Qualcomm LM80-P0436-13.

UEFI Flash Definition

Our UEFI FD starts at 0x80200000. Update your tokens in platform definition and flash definition:

And the first piece code should be your SEC initialization code (without relocation).

Little Kernel (mentioned below) will be responsible for jumping into UEFI FD at 0x80200000 and handing off execution. If you want, you can actually removes Android-specific header and device tree validation in LK (apps/aboot.c).

First-stage bootloader (Little Kernel)

DragonBoard 410c uses ARM Secure Monitor Call to switch to AArch64 mode (See Qualcomm LM80-P0436-1 for more information). The stock close-sourced SBL doesn not recognize AArch64 ELF files (later model should). LK performs basic platform initialization (UART, eMMC, MMU, etc.) A modified variant LK also initializes FrameBuffer for U-Boot. We can make it work for our UEFI too.
Windows requires UEFI provide a BGRA FrameBuffer. To achieve this, we need to modify pixel unpack pattern in platform/msm_shared/mdp5.c:


case 32:
/* Windows requires a BGRA FB */
writel(0x000236FF, pipe_base + PIPE_SSPP_SRC_FORMAT);
writel(0x03020001, pipe_base + PIPE_SSPP_SRC_UNPACK_PATTERN);

You can either specify a hard-coded address for FrameBuffer, or have a random piece of memory block to transfer information (pixel format, width, height, etc.) to UEFI. UEFI SEC phase retrieve the information, allocate HOB block and transfer information to DXE phase. A simple FrameBuffer driver retrieve information from HOB block, initializes UEFI Graphics Output Protocol. For optimal performance, initialize this piece of memory block as write-through cache memory in MMU initialization.

Persistent NVRAM Support

For persistent NVRAM support, it’s a good idea to use eMMC as storage device. This implementation demonstrates how to simulate NVRAM using eMMC and a piece of memory. I slightly modified it make it work for Qualcomm devices:

  • If eMMC NVRAM region is corrupted or uninitialized, provision it and perform a platform warm reset so I don’t get a synchronous exception in volatile variable initialization phase.
  • Modify dependency relationship to prevent “device not found” error in BlockRamVariable DXE initialization.

Windows Boot Manager depends on a “working” Real Time Clock for miscellaneous purposes. APQ8016/MSM8916 has a RTC on its PMIC processor PM8916. To access RTC services, read/write SPMI registers (see Qualcomm LM80-P0436-36). If you are lazy, just use Xen fake RTC in ArmVirtPkg.
To enable PM8916 RTC, set SPMI register 0x6046 to enabled state, then read 0x6048 and three following bits.

Note: I implemented my own PMIC protocol called PM8916Protocol that read/writes PMIC register on SPMI bus, slave #0. This RTC library is based on Xen face RTC library from ArmVirtPkg.

4KB / 64KB Page Table

For most single board computers, you will probably hit issues in ExitBootServices. EDK2 assumes runtime world follows 64KB/Page memory allocation, while most single board computers supply only less than 2GB memory. On these boards, MMU will run in 4KB PT mode. To resolve the issue, go to MdePkg/Include/AArch64/ProcessorBind.h:


///
/// The stack alignment required for AARCH64
///
#define CPU_STACK_ALIGNMENT 16

///
/// Page allocation granularity for AARCH64
///
#define DEFAULT_PAGE_ALLOCATION_GRANULARITY (0x1000)

///
/// For the sake of our SBCs
///
#define RUNTIME_PAGE_ALLOCATION_GRANULARITY (0x1000)

Set runtime page allocation granularity to 0x1000 (4KB). If your board has memory larger than 2GB, you should not modify this value; instead, check your memory allocation. There’s another interesting case with 4KB/64KB page on Cortex A53.

ARM Erratum

I randomly hit crashes (synchronous exception) during my UEFI development. After some investigation, it seems that the problem is related to load/store commands. (See ARM Errata 835769, 843419) To prevent random crashes, add these two flags to your GCC compiler:

Multi-Processor Startup (PSCI)

For platforms that implement ARM PSCI, indicate PSCI support in ACPI FADT table:


EFI_ACPI_6_0_HW_REDUCED_ACPI | EFI_ACPI_6_0_LOW_POWER_S0_IDLE_CAPABLE, // UINT32 Flags
{
EFI_ACPI_6_0_EMBEDDED_CONTROLLER,
0,
0,
EFI_ACPI_6_0_DWORD,
0x009020B4
}, // EFI_ACPI_6_0_GENERIC_ADDRESS_STRUCTURE ResetReg
1, // UINT8 ResetValue
EFI_ACPI_6_0_ARM_PSCI_COMPLIANT, // UINT16 ArmBootArchFlags
EFI_ACPI_6_0_FIXED_ACPI_DESCRIPTION_TABLE_MINOR_REVISION, // UINT8 MinorRevision

Typically you don’t need HVC call for PSCI. If you did so (and your platform doesn’t support HVC call for PSCI), you will get a INTERNAL_POWER_ERROR bugcheck with first parameter of 0x0000BEEF.
If you indicates PSCI support, you don’t have to provide parking protocol version in your ACPI MADT table. Simply set it to 0. Here’s one example:


[02Ch 0044 1] Subtable Type : 0B [Generic Interrupt Controller]
[02Dh 0045 1] Length : 50
[02Eh 0046 2] Reserved : 0000
[030h 0048 4] CPU Interface Number : 00000000
[034h 0052 4] Processor UID : 00000000
[038h 0056 4] Flags (decoded below) : 00000001
Processor Enabled : 1
Performance Interrupt Trigger Mode : 0
Virtual GIC Interrupt Trigger Mode : 0
[03Ch 0060 4] Parking Protocol Version : 00000000
[040h 0064 4] Performance Interrupt : 00000017
[044h 0068 8] Parked Address : 0000000080301000
[04Ch 0076 8] Base Address : 0000000000000000
[054h 0084 8] Virtual GIC Base Address : 0000000000000000
[05Ch 0092 8] Hypervisor GIC Base Address : 0000000000000000
[064h 0100 4] Virtual GIC Interrupt : 00000000
[068h 0104 8] Redistributor Base Address : 0000000000000000
[070h 0112 8] ARM MPIDR : 0000000000000000
[078h 0120 1] Efficiency Class : 00
[079h 0121 3] Reserved : 000000

See ARM Juno reference platform to get some idea about crafting ACPI tables.

That’s it! Welcome to Windows 10 Userland.

Spend some nights writing Windows drivers. ūüėõ

Windows 10 PE runs on a Spandragon 410 processor.
Windows 10 PE runs on a Spandragon 410 processor.

Hack TwinUI to force Windows Store Apps run on low resolution screens

Windows Store Apps on Lumia 640 XL.

Windows 8 and Windows 8.1 has a minimum screen resolution constraint for Windows Store Apps (aka. Metro Apps or whatever). If the screen resolution doesn’t meet requirement, user will see a prompt indicating the resolution is too low for these applications.

However, on certain platforms (like phones and single-board computers), it is not convenient to change resolution. Recently I am trying Windows RT 8.1 on Lumia 640 XL. Qualcomm has the resolution hard-coded in platform configuration, so I was unable to change the resolution. 1280 * 720 is not sufficient for Store Apps.

But there was an exception – the PC settings (aka. Immersive Control Panel) app. It always opens regardless of current resolution settings. So how can I force other applications to launch?

Let’s turn to TwinUI.dll. It’s one of the core components of shell infrastructure. Start IDA Pro, load TwinUI with symbols from Microsoft. Go ahead and search the existence of PC settings app. All Windows Store Apps are associated with a package family identifier. Let’s search it. In this case, it’s¬†windows.immersivecontrolpanel_cw5n1h2txyewy.

Bingo. We found it in some functions.

PC Settings Package Family ID is hardcoded in TwinUI.dll. This function has been patched by me, so it doesn't reflect actual situation you get from official Microsoft binary.
PC Settings Package Family ID is hardcoded in TwinUI.dll. This function has been patched by me, so it doesn’t reflect actual situation you get from official Microsoft binary.

By checking it’s references, we learned that layout checking routine verifies whether it is a desktop application, or PC settings app when resolution doesn’t meet requirements. Either you can patch layout checking routine or PC settings PFN verification routine. I decided to patch the second one, however patching the first is probably a better idea.

On ARMv7-A platform, I simply patched initial register store operation and the branch. Instruction BLX call was replaced with a simple NOP(MOV R0, R0).

Patched function
Patched function

There are two version of the PC settings check routines, so I need to patch both. The other one is similar to this one. Patching the layout verification routine (actually a better idea, as this patch will have some trouble when launch files from desktop) / patching on other architectures should be similar to this one.

Migrate legacy UWP project system to MSBuild-based

When Microsoft decided to adopt MSBuild on .NET Core platform, project.json was not dropped immediately until first toolchain RTM arrives. Dotnet Development on Universal Windows Platform Development leverages .NET Core too, but the depreciation progress is significantly slower than other .NET Core platforms due to historical reasons. UWP uses project.json for package management and MSBuild for builds.

In Visual Studio 2017 April Update, Microsoft finally migrates new UWP projects to full MSBuild-based project system. But our projects, which creates on early 2015, doesn’t get an auto migration as expected. Hence we decided to migrate them manually for additional benefits like better toolchain stability and advanced customization features.

Reminder: Do not attempt to use “dotnet migrate” CLI command, it won’t work for UWP projects.

Migration Prerequisites

  • Notify all your team members. Make sure everyone has Visual Studio 2017 with April update installed.
  • If you have continuous integration environment configured, make sure build agents have NuGet 4.1 or higher installed (3.5 or 4.0 won’t work).
  • Lock VCS during migration to prevent additional incidents. (We’re using TFVC for source management so that it will be easy)

Migration

  • Clean up all projects (including bin and obj directories)
  • Iterate all project directories
  • Find C# project file, open with your favorite editor.
  • Add following property group before project file lists:
<PropertyGroup>
    <RestoreProjectStyle>PackageReference</RestoreProjectStyle>
</PropertyGroup>

Okay, you’ve completed the first step. Then open your project.json file. Migrate all NuGet packages references as the picture below.

Package Reference
Package Reference

Finally, remove project.json and additional files like project.lock.json, *.nuget.targets, *.nuget.props. (Or your will get lots of warning that may lead .NET Native compilation fail)

Do it for every project. Then open Visual Studio, restore NuGet packages for all projects, build to validate and submit changes.

The Windows “Gatekeeper” Internals

"Rickrolling" in Windows SmartScreen

Windows 10 Insider Preview 15046 introduces the Windows-flavor “Gatekeeper“. It is similar to Gatekeeper in macOS, with some minor differences.

First of all, Windows “Gatekeeper” doesn’t block the execution of applications that don’t require installation. I tried to run PuTTY, a popular¬†tool on Windows and it works.

Secondly, Windows “Gatekeeper” is based on¬†Microsoft SmartScreen, which means disabling SmartScreen will turn it off too. Prior to application execution, SmartScreen will send file hash and publisher information(including certificate thumbprint) to Microsoft’s server, then SmartScreen server send back metadata including application reputation. Response is signed with a specific key that will be checked in client side for message integrity.

Unlike macOS, attempt to start application from console(e.g. Command Prompt and PowerShell) will trigger “Gatekeeper”.

Attempt to start application from PowerShell
Attempt to start application from PowerShell

The window is web-based. Although you can’t modify the response directly(no one wants to deal with sha256RSA unless the key leaks), you can attach a debugger to have some fun with it.

"Rickrolling" in Windows SmartScreen
“Rickrolling” in Windows SmartScreen

Microsoft claims that this feature is opt-in for most Windows SKUs (except Windows 10 Cloud AFAIK), and it is not revalent to UMCI (User-mode Code Integrity), which is enforced in Windows 10 Cloud.

A Simple Na√Įve ™ Guide to ASP.NET Core Linux Deployment

Step 1: Prepare .NET Core environment

This step is absolutely easy on Ubuntu/Debian and other officially supported distributions. If you are using Archlinux, go to AUR for .NET Core runtime.

Step 2: Prepare files.

In local Visual Studio or other development environment, publish project to a folder. (dotnet build -c Release)

Copy files.

Step 3: Setting services

I wrote a simple systemd service for my application:

[Unit]
Description=A certain ASP.NET Core application
After=network.target

[Service]
User=www-data
Group=www-data
WorkingDirectory=/opt/imbushuo/somepath
ExecStart=/usr/bin/dotnet /opt/imbushuo/somepath/App.dll --server.urls=http://localhost:5050
Restart=on-failure
RestartSec=5
Environment="ASPNETCORE_ENVIRONMENT=Production"

[Install]
WantedBy=multi-user.target

Start it.

Step 4: Setting up reverse proxy

Refer to your frontend server documentation for details.

Some hints:

If you are using per-environment configuration file, make sure the configuration starts with capital letter, like appsettings.Production.json . Otherwise, the Startup class will not load the settings file.

If you want to run multiple applications, make sure add configuration class in Program class and apply it. Then, pass server.urls parameter with address and port, like what I wrote above.

The following Program.cs has been modified to enable parameter support. Feel free to copy it:

using System.IO;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Configuration;

namespace HomePortal
{
    public class Program
    {
        public static void Main(string[] args)
        {
            var configuration = new ConfigurationBuilder()
            .AddCommandLine(args)
            .Build();

            var host = new WebHostBuilder()
                .UseKestrel()
                .UseContentRoot(Directory.GetCurrentDirectory())
                .UseConfiguration(configuration)
                .UseIISIntegration()
                .UseStartup<Startup>()
                .Build();

            host.Run();
        }
    }
}

Fixing Single Sign Out for Auth0 WordPress Integration

Since I set up Active Directory & Azure Active Directory for my workgroup and myself, I decided to switch to SSO for my web services. I chose Auth0 as the WordPress Identity Middleware as it is pretty flexible. However, the log out function, doesn’t work properly on federated logons. It will just call auth0 to sign out instead of signing out of all IdPs.

Luckily, fixing that is pretty easy. Located to lib/WP_Auth0_LoginManager.php and find the logout function:

If you don’t see federated after logout, fix it. They have fixed it on GitHub, but I don’t know why they don’t push it to WordPress release.

DragonBoard 410c w/ UEFI

DragonBoard 410c runs Windows RT 8.1

I purchased a Qualcomm DragonBoard 410c for some specific purpose (generally, for fun). All DragonBoard 410c ship with Android, but I want Windows IoT and UEFI so I flashed it.

Dragonboard 410c runs Windows RT 8.1
Dragonboard 410c runs Windows RT 8.1

Well, you can boot Windows RT 8.1 on it, as long as you got critical HAL extensions. But the most important one, USB controller, is not available on DragonBoard 410c. It utilizes USB Role Switch, which is officially supported in Windows 10, not in Windows 8.1. I posted how to boot Windows RT 8.1 installation disk on XDA Developers forum, if you are interested in that, please search it.

Note: USB will not be a problem for DragonBoard 800 because Snapdragon 800 has more than one USB channel, while Snapdragon 410 has only one USB channel.

Windows failed to load with Bugcheck code 0x5c
Windows failed to load with Bugcheck code 0x5c

I think there’s something weird with the internal EFI shell. It seems that all GOP operations will let the firmware hang, however, you can run EFI applications with GOP in the internal EBL(yet another lightweight EFI shell). What’s more, if you boot any Windows OS releases from EFI shell, it will crash during HAL initialization. Such situation doesn’t exist on any Qualcomm-based Windows Phones (MSM8960), so I believe it is a bug specific to 410c’s firmware.

Huaji PC Boot Logo
Huaji PC Boot Logo

All ACPI-related files, including boot logo, is stored in a small hidden FAT16 partition called PLAT. You can replace files – but I haven’t tested customized ACPI DSDT table. Maybe we can let USB controller work by removing device URS0 and expose device USB0 to root.

GRUB-EFI (ARMHF) works on Qemu emulator, but it will hang on 410c’s firmware. I haven’t got a USB UART cable yet, so I didn’t know what happened.

Rule-based Routing & Traffic Forwarding with IPsec Site to Site VPN and Linux

This article is adapted from https://www.v2ex.com/t/180070 and http://hjc.im/shi-yong-strongswanda-jian-ipsecikev2-vpn/. The major difference is ShadowVPN is replaced by StrongSwan IPsec VPN in this article.

Readiness Check

  • Upgrade your staging/production environment to the latest version. Make sure all security patches are installed.
  • Make sure you have packages libpam0g-dev libssl-dev make gcc installed.

Building StrongSwan

Download the latest version here: http://download.strongswan.org/strongswan.tar.gz

Unarchive it, and configure using the following params:

./configure  --enable-eap-identity --enable-eap-md5 
--enable-eap-mschapv2 --enable-eap-tls --enable-eap-ttls --enable-eap-peap  
--enable-eap-tnc --enable-eap-dynamic --enable-eap-radius --enable-xauth-eap  
--enable-xauth-pam  --enable-dhcp  --enable-openssl  --enable-addrblock --enable-unity  
--enable-certexpire --enable-radattr --enable-tools --enable-openssl --disable-gmp --enable-kernel-libipsec

We have to specify routing table’s priority on client server(not IPsec Access Server)¬†since we wants to specify routing table manually: –with-routing-table-prio=32800.¬†¬†Also, TAP/TUN device is enabled instead of StrongSwan’s own kernel module. It will simplify the configuration later.

Then make and install it.

Configure IPsec Access Server

Just a reminder, if you have any issues about IP range, please refer to the demo topology graph. The image is unavailable right now for some reason. I’ll fix it ASAP.

Go to /usr/local/etc. Edit ipsec.secrets:

: PSK "<IPsec PRE SHARED KEY, PLEASE REMEMBER TO REPLACE IT WITH YOU OWN KEY>"
s2s : XAUTH "<ANY PASSWORD YOU WANT, PLEASE REMEMBER TO REPLACE IT WITH YOU OWN PASSWORD>"

Go to /usr/local/etc. Edit ipsec.conf:

config setup
    uniqueids=never 

conn s2sbj1_xauth_psk
    keyexchange=ikev1
    left=%defaultroute
    leftauth=psk
    leftsubnet=0.0.0.0/0
    right=%any
    rightauth=psk
    rightauth2=xauth
    rightsourceip=100.11.2.0/24
    auto=add

Go to /usr/local/etc. Edit strongswan.conf:

 charon {
         load_modular = yes
         duplicheck.enable = no
         compress = yes
         plugins {
                 include strongswan.d/charon/*.conf
         }
         # In China, please consider about replacing 8.8.8.8/8.8.4.4 to 114.114.114.114. They do offer correct DNS query results outside mainland China, as long as you have configured the Chinese routing exception for it(route to non-mainland China outbound server)
         dns1 = 8.8.8.8
         dns2 = 8.8.4.4
         nbns1 = 8.8.8.8
         nbns2 = 8.8.4.4
 }
 include strongswan.d/*.conf

Turn on IPv4 forwarding in sysctl.conf.

Edit iptables. The following configuration is for Ubuntu 14.04 LTS. For other Linux distro, check out its documentation.

iptables -A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -s 100.11.0.0/24  -j ACCEPT
iptables -A FORWARD -s 100.11.1.0/24  -j ACCEPT
iptables -A FORWARD -s 100.11.2.0/24  -j ACCEPT
iptables -A INPUT -i eth0 -p esp -j ACCEPT
iptables -A INPUT -i eth0 -p udp --dport 500 -j ACCEPT
iptables -A INPUT -i eth0 -p tcp --dport 500 -j ACCEPT
iptables -A INPUT -i eth0 -p udp --dport 4500 -j ACCEPT
iptables -A INPUT -i eth0 -p udp --dport 1701 -j ACCEPT
iptables -A INPUT -i eth0 -p tcp --dport 1723 -j ACCEPT
iptables -A FORWARD -j REJECT
iptables -t nat -A POSTROUTING -s 100.11.0.0/24 -o eth0 -j MASQUERADE
iptables -t nat -A POSTROUTING -s 100.11.1.0/24 -o eth0 -j MASQUERADE
iptables -t nat -A POSTROUTING -s 100.11.2.0/24 -o eth0 -j MASQUERADE

Save it

iptables-save > /etc/iptables.rules
cat > /etc/network/if-up.d/iptables<<EOF
#!/bin/sh
iptables-restore < /etc/iptables.rules
EOF
chmod +x /etc/network/if-up.d/iptables

Configure IPsec client

Suppose you have the correct StrongSwan with TAP/TUN and routing table priority installed.

Go to /usr/local/etc, edit ipsec.secrets, just put what you have in the previous step.

Go to /usr/local/etc, edit ipsec.conf:

config setup
    uniqueids=never 
    strictcrlpolicy=no
    

conn s2sbj1
    keyexchange=ikev1
    ikelifetime=1440m
    keylife=120m
    # Enable this will cause authentication failure
    aggressive=no
    ike=aes-sha2-modp2048
    esp=ase-sha2
    xauth=client
    left=<eth0's IP address>
    # Ask your server
    leftsourceip=%config
    leftauth=psk
    leftauth2=xauth
    rightauth=psk
    rightauth2=xauth
    right=<Your IPsec VPN Server's public IP address>
    # For Microsoft Azure and other service providers who use SNAT, specify that to prevent IKE_SA failure
    rightid=%any
    xauth_identity=s2s
    auto=add
    rightsubnet=0.0.0.0/0

Establish Connection

On your IPsec VPN server, type sudo ipsec start .

On your client server, type:

sudo ipsec start
sudo ipsec up s2sbj1

It should get connected shortly. Go to ifconfig and you should find a new network adapter called ipsec0.

Configure rule-based routing

Create a new routing table.

user@ibntwkstgepbj1:~$  sudo vim /etc/iproute2/rt_tables
200 bj1s2s
:q
user@ibntwkstgepbj1:~$

Get the routing configuration in table 220 (IPsec table).

user@ibntwkstgepdm1:~$  sudo ip route list table 220
default dev ipsec0  proto static  src 100.11.2.1
42.159.66.233 via 10.0.0.1 dev eth0  proto static  src 10.0.0.4

Specify the default route for this table (copy it from 220):

user@ibntwkstgepdm1:~$  sudo ip route add default dev ipsec0  proto static  src 100.11.2.1 table bj1s2s

(You don’t have to copy the second line I guess, but I added that)

Add IP rules.

user@ibntwkstgepdm1:~$  ip rule add from <IP Range> table bj1s2s

Refresh routing table.

user@ibntwkstgepdm1~$  ip route flush cache

Compose a shell script if you want to compelte that automatically for every reconnection.

Conclusion

We implemented a simple line optimization using rule-based routing in this example. For application-based service, you are all set and ready to go. For VPN access services, configuration for iptables is needed in order to tag data packets and route them correctly. Check out this article for more details.

I didn’t offer a auto routing script in this example. I strongly recommend you to write it since it saves your time by configuring routing table automatically.

For multiple IPsec connections and routings, just specify the source IP, which is 100.11.2.1 and 100.21.2.1 in routing tables. They use the same adapter.