[Swan] Forwarding between multiple LibreSwan/IPSEC tunnels

Paul Wouters paul at nohats.ca
Sat Nov 26 13:16:25 UTC 2016

On Wed, 23 Nov 2016, Jason Martin wrote:

> Subject: [Swan] Forwarding between multiple LibreSwan/IPSEC tunnels

> In this scenario there are two VPCs being connected, and a
> instance that happens to be in a 3rd VPC is performing the
> routing and acting as a hub.

> My issue is that the hub is able to reach both East and West,
> but packets from either end arrive on hub but reach no further.

If you configured tunnels properly (from east to west terminating
at the middle box) then my guess is firewalling, NATing, forwarding
or rp_filter to be the culprit.

> I notice that counters 'XfrmInNoPols' increments  when I attempt to ping.
> I suspect the issue is somewhere in the XFRM config, but its not
> clear to me how to resolve the issue (if that is even the
> issue).
> Topology:
> West ( - (hub) - East ( Hub is
> connecting to both ends via VGW's, so cleartext packets for
> east/west never leave Hub. As per normal VGW behavior, two
> tunnels exist between each end and HUB.

west should have a conn with hub:

conn tunnel

east should have a similar conn with hub. So east and west
have 1 tunnel, and hub has two tunnels. Is that what you have?

> The basis for this configuration is
> https://github.com/patrickbcullen/Openswan-VPC, modified to
> support a 2nd set of tunnels. One oddity about this script is it
> set ups a 'network namespace'
> (http://man7.org/linux/man-pages/man8/ip-netns.8.html) to handle
> all the ipsec and routing.

Seems a bit overengineered :)
If using namespaces, you might have to ensure that the forwarding and
rp_filter and NAT iptables rules are properly setup for that namespace
too. I'm a little confused how you would have the hubip in the
configuration within the special namespace if it is also the main IP
for the host itself?

> The hub can ping nodes in east and west via the IPSEC tunnels.
> The VGW's agree that ipsec and BGP is up, the the East/West
> subnets see the propagated routes. The hub has routes to both
> East and West. Iptables is fully open. rp_filter is set to 0 and
> forwarding / ip_forward is set to 1 in sysctl.

Run "ipsec verify" to confirm it all is set as expected?

> I set up a ping generator in West that is attempting to ping
> East. The packets reach the openswan network namespace in hub:
> 16:38:49.311665 IP >
> ESP(spi=0x0a790d98,seq=0x4f5), length 132
> 16:38:49.311665 IP > ICMP echo
> request, id 411, seq 1113, length 64
> I have NFLOG / ulogd2 setup in iptables. It shows:
> RAW-PREROUTING IN=eth0 OUT= MAC=d6:fd:61:4b:73:42:6a:3a:bb:e2:33:75:08:00 SRC= DST= LEN=84 TOS=00 PREC=0x00 TTL=254 ID=49803 DF PROTO=ICMP TYPE=8 CODE=0 ID=411 SEQ=1155 MARK=0
> NAT-PREROUTING IN=eth0 OUT= MAC=d6:fd:61:4b:73:42:6a:3a:bb:e2:33:75:08:00 SRC= DST= LEN=84 TOS=00 PREC=0x00 TTL=254 ID=49803 DF PROTO=ICMP TYPE=8 CODE=0 ID=411 SEQ=1155 MARK=0
> However the packet never reaches the FORWARD iptables chain:
> Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
> Pinging from East to West fails similarly.
> The hub can ping both the source and destination:
> # ping -c 1
> 64 bytes from icmp_seq=1 ttl=254 time=1.74 ms
> # ping -c 1
> 64 bytes from icmp_seq=1 ttl=254 time=94.3 ms
> Any suggestions on what might be blocking packets from
> transiting hub? I've seen hints that 'eroute' might be filtering
> the packets, but when I try to look at the eroute table with
> 'ipsec eroute' I get an error that NETKEY and eroute aren't
> compatibile.

You can check using "ip xfrm pol" and "ip xfrm state" what the
kernel state is. You'd have to run this within the namespace.

Oh, you included the barf which has the xfrm settings

> + ip xfrm policy
> src dst

Not sure why you have 169.254 ipsec policies? Either you lied
and rewrote the above IP addresses from the real ones used,
or you are surely missing IPsec policies in this namespace?
Or perhaps barf was run in the wrong namespace?

> XfrmInTmplMismatch      	8649

Well, 169.254 does not match 172.16 but how could you get
the wrong policies? Possibly namespace confusion?

> # begin conn awstunnel1
> conn awstunnel1
> 	left=
> 	leftid=""
> 	leftsubnet=
> 	left=
> 	right=
> 	rightid=""
> 	rightsubnet=
> 	right=

So my guess is you rewrote some IP addresses in your email?

I guess I was expecting you wanted to connect two subnets to
each other but you are using so that means you want
to extrude a subnet ? That is the subnets are given connectivity
to the world, and not just to each other? That does not match
what you were trying to do as written in your email?


More information about the Swan mailing list