[Swan] Advice on troubleshooting AWS VPC site-to-site VPN connection.

[Paul Wouters]

On Thu, 14 Oct 2021, Scott Classen wrote:

> Now I have an EC2 instance that is associated with my AWS VPC with a public and private IP address
>
> Private: 10.0.1.252
> Public: xxx.xxx.79.208
>
> I can ping the public address no problem, but when I ping the private address the pings appear to go out as UDP-encap: ESP packets (I think?) but they return from the public address! See this tcpdump output:
>
> # ping -c3 10.0.1.252
> PING 10.0.1.252 (10.0.1.252) 56(84) bytes of data.
> 64 bytes from xxx.xxx.79.208: icmp_seq=1 ttl=240 time=3.44 ms
> 64 bytes from xxx.xxx.79.208: icmp_seq=2 ttl=240 time=5.13 ms
> 64 bytes from xxx.xxx.79.208: icmp_seq=3 ttl=240 time=3.46 ms


So this looks good. The ping is using the internal IP and gets
a response on the internal IP

> # tcpdump -ni enp2s0f0 udp port 500 or udp port 4500 or icmp
> 13:30:54.113295 IP xxx.xxx.85.19.ipsec-nat-t > 52.9.186.125.ipsec-nat-t: UDP-encap: ESP(spi=0xc9530708,seq=0x20), length 132
> 13:30:54.116788 IP xxx.xxx.79.208 > xxx.xxx.85.19: ICMP echo reply, id 56943, seq 1, length 64
> 13:30:55.114989 IP xxx.xxx.85.19.ipsec-nat-t > 52.9.186.125.ipsec-nat-t: UDP-encap: ESP(spi=0xc9530708,seq=0x21), length 132
> 13:30:55.118301 IP xxx.xxx.79.208 > xxx.xxx.85.19: ICMP echo reply, id 56943, seq 2, length 64
> 13:30:56.116497 IP xxx.xxx.85.19.ipsec-nat-t > 52.9.186.125.ipsec-nat-t: UDP-encap: ESP(spi=0xc9530708,seq=0x22), length 132
> 13:30:56.119892 IP xxx.xxx.79.208 > xxx.xxx.85.19: ICMP echo reply, id 56943, seq 3, length 64

This looks good a little odd. The fact you see decrypted traffic is not
wrong. The reason you see this is an artifact of how the linux kernel
implements the hooks which tcpdump uses to see traffic.  Basically,
it sees outgoing traffic after encrypt, and incoming traffic after
decrypt. If you tcpdumped on the machine in front of your linux machines,
you would see traffic as UDP-encap: ESP both ways.

You can run "ipsec trafficstatus" and look at the byte counters to
confirm traffic is properly encrypted in both directions.

You can also check 'cat /proc/net/xfrm_stat' to see any problems. All
the counters in this file should be 0.

Paul