StrongSwan14. Apr '17

Following assumes X509 infrastructure is in place and we can easily set up certificates for the services. Even if you're familiar with OpenVPN there are a lot of things done differently for IPsec. For instance IP packets are encrypted usually by the kernel instead of an userspace process. The latter is mainly concerned about the initial authentication of endpoints, exchange of keying material, traffic selectors.

This very often means that you can't for example install hardware enablement stack provided by Ubuntu as this will upgrade the kernel, but not the userspace components. As a workaround StrongSwan includes libipsec plugin which implements kernelspace components as a library and uses TUN interface to talk to the OS making it very similar to OpenVPN on the expense of performance degradation.

In IPSec jargon two payload modes are possible:

Transport mode encrypts the payload leaving packet headers (source and destination address, extensions) intact. This is suitable for securing communications between publicly routable IP addresses (eg between servers or connecting to a server) or securing communications inside LAN.
Tunnel mode encrypts the whole packet and attaches new headers. This is suitable for connecting private IPv4 subnets over public Internet or connecting road warriors to corporate network. In case of roadwarriors virtual IP-s are assigned by the IPSec gateway a'la DHCP.

Note that tunnel mode is also very similar to the way OpenVPN operates while transport mode is more closer to SSL/TLS.

StrongSwan has a lot of constraints when it comes to gateway certificates:

DNS subject alternative name extension has to be added
ikeIntermediate (1.3.6.1.5.5.8.2.2) extended key usage flag has to be added
Subject key identifier
Authority key identifier

Important

On Fedora StrongSwan stuff is placed under /etc/strongswan instead of /etc

libipsec plugin

Thus the very first step to take when you see something not working as expected:

apt install strongswan-plugin-kernel-libipsec # Ubuntu
dnf install strongswan-libipsec # Fedora

Site to site tunnel

Following is /etc/ipsec.conf for the passive/listening endpoint, it is assumed that this machine is accessible from the internet:

conn site-to-site
    auto=add
    left=router1.koodur.com
    right=router2.koodur.com
    leftfirewall=yes
    leftsubnet=192.168.12.0/24
    rightsubnet=192.168.72.0/24

Following is /etc/ipsec.conf for the active/connecting endpoint, it could be behind NAT, note that right= is missing in this case:

conn site-to-site
    auto=start
    left=stream.koodur.com
    leftfirewall=yes
    right=robo-router.koodur.com
    leftsubnet=192.168.72.0/24
    rightsubnet=192.168.12.0/24

To establish the connection you need to open up 500 and 4500 ports on the gateway:

iptables -I INPUT -p esp -j ACCEPT
iptables -I INPUT -p udp --dport 500 -j ACCEPT
iptables -I INPUT -p udp --dport 4500 -j ACCEPT

The way IPsec is implemented in the kernel it will seem like the packets coming from the tunnel are coming from the WAN interface. This makes it trickier to implement firewall rules, the packets coming from a tunnel are distinguished by -m policy --dir in --pol ipsec rule. To trust all packets coming from a tunnel:

iptables -I INPUT -m policy --pol ipsec --dir in -j ACCEPT
iptables -I FORWARD -m policy --pol ipsec --dir in -j ACCEPT

As packets leaving on the WAN interface are usually NAT-ed it will also mangle the packets intended to go to the tunnel. To prevent NAT-ing those packets:

iptables -t nat -I POSTROUTING  -m policy --dir out --pol ipsec -j ACCEPT

On-demand routing

IPSec gateway

Setting up Windows clients

Windows 10 and probably older ones as well have built-in VPN client capable of speaking IKEv2, that's the key exchange protocol used by IPSec.

Hardening ciphers

StrongSwan distinguishes key exchange cipherset (IKE) and ciphers used to hash and encrypt packages (ESP).

Windows 10 clients offer following cipher suites:

IKE:3DES_CBC/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_1024
IKE:AES_CBC_256/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_1024
IKE:3DES_CBC/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_1024
IKE:AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_1024
IKE:3DES_CBC/HMAC_SHA2_384_192/PRF_HMAC_SHA2_384/MODP_1024
IKE:AES_CBC_256/HMAC_SHA2_384_192/PRF_HMAC_SHA2_384/MODP_1024
ESP:AES_CBC_256/HMAC_SHA1_96/NO_EXT_SEQ
ESP:3DES_CBC/HMAC_SHA1_96/NO_EXT_SEQ

To disable 3DES and retain compatilibity with Windows 10:

ike=aes256-sha384-modp1024!
esp=aes256-sha1!

Debugging

IPSec encryption happens between application transmitting a packet and leaving the network interface.

In case of To list policies used to encapsulate internal addresses:

ip xfrm policy

To list symmetric keys used to encrypt the packets:

ip xfrm state

In case of

OpenVPN StrongSwan IPSec iptables