[Swan] Advice on troubleshooting AWS VPC site-to-site VPN connection.

[Scott Classen]

OK Have moved to a machine running CentOS8 so I can use a newer Libreswan version (Linux Libreswan 4.3 (netkey) on 4.18.0-305.19.1.el8_4.x86_64).

I am also only bringing up a only single VPN tunnel for the time being since the right side (AWS) has two tunnel endpoints in different subnets. My current connection is:

conn conn-to-aws-1
    leftid=xxx.xxx.85.19
    left=xxx.xxx.85.19
    leftsubnets=192.168.1.0/24,xxx.xxx.85.19/32
    rightid=xxx.xxx.186.125
    right=xxx.xxx.186.125
    rightsubnets=xxx.xxx.63.92/30,10.0.1.0/24
    authby=secret
    ikev2=insist
    #aggressive=no
    ikelifetime=28800
    lifetime=3600
    rekey=yes
    rekeyfuzz=100%
    fragmentation=yes
    replay-window=1024
    dpddelay=30
    dpdtimeout=120
    mtu=1419
    type=tunnel
    auto=start
    ike=aes128-sha1-modp1024
    esp=aes128-sha1-modp1024
    keyingtries=%forever


# ip xrfm policy

Returns a lot of seemingly important routing information like this:


src 10.0.1.0/24 dst 192.168.1.0/24 
	dir fwd priority 2084814 ptype main 
	tmpl src xxx.xxx.186.125 dst xxx.xxx.85.19
		proto esp reqid 16393 mode tunnel
src 10.0.1.0/24 dst 192.168.1.0/24 
	dir in priority 2084814 ptype main 
	tmpl src xxx.xxx.186.125 dst xxx.xxx.85.19
		proto esp reqid 16393 mode tunnel


Now I have an EC2 instance that is associated with my AWS VPC with a public and private IP address

Private: 10.0.1.252
Public: xxx.xxx.79.208

I can ping the public address no problem, but when I ping the private address the pings appear to go out as UDP-encap: ESP packets (I think?) but they return from the public address! See this tcpdump output:

# ping -c3 10.0.1.252
PING 10.0.1.252 (10.0.1.252) 56(84) bytes of data.
64 bytes from xxx.xxx.79.208: icmp_seq=1 ttl=240 time=3.44 ms
64 bytes from xxx.xxx.79.208: icmp_seq=2 ttl=240 time=5.13 ms
64 bytes from xxx.xxx.79.208: icmp_seq=3 ttl=240 time=3.46 ms


# tcpdump -ni enp2s0f0 udp port 500 or udp port 4500 or icmp
13:30:54.113295 IP xxx.xxx.85.19.ipsec-nat-t > 52.9.186.125.ipsec-nat-t: UDP-encap: ESP(spi=0xc9530708,seq=0x20), length 132
13:30:54.116788 IP xxx.xxx.79.208 > xxx.xxx.85.19: ICMP echo reply, id 56943, seq 1, length 64
13:30:55.114989 IP xxx.xxx.85.19.ipsec-nat-t > 52.9.186.125.ipsec-nat-t: UDP-encap: ESP(spi=0xc9530708,seq=0x21), length 132
13:30:55.118301 IP xxx.xxx.79.208 > xxx.xxx.85.19: ICMP echo reply, id 56943, seq 2, length 64
13:30:56.116497 IP xxx.xxx.85.19.ipsec-nat-t > 52.9.186.125.ipsec-nat-t: UDP-encap: ESP(spi=0xc9530708,seq=0x22), length 132
13:30:56.119892 IP xxx.xxx.79.208 > xxx.xxx.85.19: ICMP echo reply, id 56943, seq 3, length 64


Is this a Libreswan/routing problem or an AWS routing problem.


BTW I disabled firewalld while running these tests and IPsec verify seems happy.

Thanks,
Scott