[Swan] VTI for Multiple Site-to-Site VPNs

Wed Aug 28 14:55:36 UTC 2019

Hi,

I've recently made some changes to my IPSec setup and after these 
changes it seems things aren't working as expected.

Previously I had a single site-to-site VPN, with a Cisco IOS router on a 
dynamic CGNAT connection connecting to my libreswan head end.  It works 
fine as a route based VPN and I have a numbered link /30 that I run 
between the VTI and Cisco IOS Tunnel interface.

This worked well.

But things have gone wrong when I've attempted to add more VTI sessions 
to the libreswan side.

Configs look like this:

conn router-2.reub.net-ipv4
         left=43.229.60.170
         leftid=@jetstream.reub.net
         leftsubnet=0.0.0.0/0
         right=%any
         rightid=router-2 at reub.net
         rightsubnet=0.0.0.0/0
         authby=secret
         ikev2=insist
         ikelifetime=86400s
         salifetime=3600s
         # IOS XE
         ike=aes-sha2_512;dh19
         # Classic IOS
         #ike=aes-sha2_512;dh5
         dpddelay=15
         dpdtimeout=45
         dpdaction=clear
         auto=add
         mark=1/0xffffffff
         vti-interface=vti-1
         leftvti=192.168.6.33/30

I've attempted to add additional configuration by cloning the config 3x 
above for 3 additional devices, pulling it in via an include file and 
changing:

- the conn name to be unique for each
- the rightid value (to match what the remote so libreswan can 
identify/match each peer)
- incrementing the mark value from 1 upwards leaving the mask the same
- incrementing the vti-interface value from vti-1 to vti-2/3/4 etc
- changing the leftvti IP address by adding +4 to each subnet to create 
a separate unique /30 for each VTI

It all loads up fine when libreswan is started.

However this configuration doesn't pass traffic for additional remotes. 
Often the first one to connect is able to.  IPSec seems to go through to 
completion and the two ends see IKEv2 and SA's all looking fine, yet 
there is no connectivity across the VTI's.

There are errors logged like this:

Aug 29 00:12:12.882876: "router.x-ipv4"[48] 180.150.x.x #853: up-client 
output: vti interface "vti-2" already exists with conflicting setting 
(perhaps need vti-sharing=yes ?
Aug 29 00:12:12.894613: "router.x-ipv4"[48] 180.150.x.x #853: 
prepare-client output: vti interface "vti-2" already exists with 
conflicting setting (perhaps need vti-sharing=yes ?

and

Aug 29 00:12:12.870911: "router.x.x-ipv4"[48] 180.150.x.x #853: WARNING: 
connection router.x.x-ipv4 PSK length of 20 bytes is too short for 
sha2_512 PRF in FIPS mode (32 bytes required)

[But I'm not running in FIPS mode and I don't care about FIPS, I've got 
a 20+ character PSK already]

and these, which I think are the big problem and point to the real root 
cause:

Aug 29 00:11:27.293077: "router-2.x.x-ipv4"[289] 118.148.x.x #849: 
up-client output: add tunnel "ip_vti0" failed: File exists
Aug 29 00:11:27.294102: "router-2.x.x-ipv4"[289] 118.148.x.x #849: 
up-client output: sysctl: cannot stat 
/proc/sys/net/ipv4/conf/vti-4/disable_policy: No such file or directory
Aug 29 00:11:27.294825: "router-2.x.x-ipv4"[289] 118.148.x.x #849: 
up-client output: sysctl: cannot stat 
/proc/sys/net/ipv4/conf/vti-4/rp_filter: No such file or directory
Aug 29 00:11:27.295534: "router-2.x.x-ipv4"[289] 118.148.x.x #849: 
up-client output: sysctl: cannot stat 
/proc/sys/net/ipv4/conf/vti-4/forwarding: No such file or directory
Aug 29 00:11:27.298539: "router-2.x.x-ipv4"[289] 118.148.x.x #849: 
up-client output: Cannot find device "vti-4"
Aug 29 00:11:27.301150: "router-2.x.x-ipv4"[289] 118.148.x.x #849: 
up-client output: Cannot find device "vti-4"
Aug 29 00:11:27.314097: "router-2.x.x-ipv4"[289] 118.148.x.x #849: 
prepare-client output: add tunnel "ip_vti0" failed: File exists
Aug 29 00:11:27.314890: "router-2.x.x-ipv4"[289] 118.148.x.x #849: 
prepare-client output: sysctl: cannot stat 
/proc/sys/net/ipv4/conf/vti-4/disable_policy: No such file or directory
Aug 29 00:11:27.315655: "router-2.x.x-ipv4"[289] 118.148.x.x #849: 
prepare-client output: sysctl: cannot stat 
/proc/sys/net/ipv4/conf/vti-4/rp_filter: No such file or directory
Aug 29 00:11:27.316386: "router-2.x.x-ipv4"[289] 118.148.x.x #849: 
prepare-client output: sysctl: cannot stat 
/proc/sys/net/ipv4/conf/vti-4/forwarding: No  such file or directory
Aug 29 00:11:27.318860: "router-2.x.x-ipv4"[289] 118.148.x.x #849: 
prepare-client output: Cannot find device "vti-4"
Aug 29 00:11:27.321272: "router-2.x.x-ipv4"[289] 118.148.x.x #849: 
prepare-client output: Cannot find device "vti-4"
Aug 29 00:11:27.321843: "router-2.x.x-ipv4"[289] 118.148.x.x #849: 
prepare-client command exited with status 1

The conf file for that device with the matching conn has in it 
vti-interface=vti-4 but it is failing to get created:

jetstream /etc/ipsec.d # ip tunnel
tunl0: any/ip remote any local any ttl inherit nopmtudisc
ip_vti0: ip/ip remote any local any ttl inherit nopmtudisc key 0
vti-2: ip/ip remote any local 43.229.60.170 ttl inherit key 2
jetstream /etc/ipsec.d #

Where does that reference to ip_vti0 come from as I do not have that 
defined anywhere in any of the device config files?

jetstream /etc/ipsec.d # grep ip_vti0 *
grep: policies: Is a directory
grep: temp: Is a directory
jetstream /etc/ipsec.d #

The end goal that I am trying to do and expecting to see is 4 separate 
interfaces vti-1 through vti-4, one for each remote peer, and be able to 
ping across them as directly connected routes.  Once that's working I 
can worry about routing of non directly connected subnets.

The system is a Gentoo Linux VM, running libreswan -git as of a few days 
ago.

What am I missing here?

Thanks,
Reuben