[Swan] ***UNCHECKED*** Re: Traffic _stops_ routing down tunnel

Paul Wouters paul at nohats.ca
Wed Apr 8 21:32:11 EEST 2015


On Fri, 3 Apr 2015, Paul Moore wrote:

> Do you have any insight on this one?

Not really? The problem you describe sounds more like a firewall thing,
like a RELATED rule that kicks in only once traffic flows in one
direction. IPsec policies are very symmetric, so if traffic can flow
then that part is just properly installed and working.

Try disabling firewall rules and stuff?

recheck with "ipsec verify" to see if there are any warnings?

Paul

> I've been playing with accept_redirects, send_redirects, secure_redirects and rp_filter and have turned everything off, but it doesn't
> change anything.
> 
> I'm limping along for now by sending syslog traffic from the AWS host to the other end to keep the ability of the other end to ping the
> AWS host, but it's not really regular enough to be reliable.
> 
> Cheers,
> 
> Paul MooreAstute Systems
> pmoore at astute-systems.com      0481 268 522       [btn_in_20x15.png] View my profile
> 
> On 20 March 2015 at 15:57, Paul Moore <pmoore at astute-systems.com> wrote:
>       Hi Paul,
> Thanks for your reply.
> 
> Before changing any configuration, I've pasted the output of the ipsec initiator at http://pastebin.com/ct532Ewc and the ipsec
> responder at http://pastebin.com/1bzGcq1d
>
>             Does the server have one interface in the 10.1.2.0/24 network? If so,
>             can you add leftsourceip=10.1.2.x to that? (where 10.1.2.x is the
>             IP the server has in the 10.1.2.0/24 network?)
> 
> 
> The ipsec initiator (mdserver) has it's only ethernet address as 10.1.2.2/24 on the internal network.
>
>       [root at mdserver ~]# ifconfig
> enp0s29f0u2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>         ether 02:21:5e:0a:a9:1f  txqueuelen 1000  (Ethernet)
>         RX packets 39263  bytes 2555803 (2.4 MiB)
>         RX errors 0  dropped 0  overruns 0  frame 0
>         TX packets 0  bytes 0 (0.0 B)
>         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
> 
> enp11s0f0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>         inet 10.1.2.2  netmask 255.255.255.0  broadcast 10.1.2.255
>         inet6 fe80::221:5eff:fe09:a91c  prefixlen 64  scopeid 0x20<link>
>         ether 00:21:5e:09:a9:1c  txqueuelen 1000  (Ethernet)
>         RX packets 184092  bytes 41376693 (39.4 MiB)
>         RX errors 0  dropped 29  overruns 0  frame 0
>         TX packets 138809  bytes 43058412 (41.0 MiB)
>         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
> 
> enp11s0f1: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
>         ether 00:21:5e:09:a9:1e  txqueuelen 1000  (Ethernet)
>         RX packets 0  bytes 0 (0.0 B)
>         RX errors 0  dropped 0  overruns 0  frame 0
>         TX packets 0  bytes 0 (0.0 B)
>         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
> 
> lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
>         inet 127.0.0.1  netmask 255.0.0.0
>         inet6 ::1  prefixlen 128  scopeid 0x10<host>
>         loop  txqueuelen 0  (Local Loopback)
>         RX packets 165379  bytes 52434556 (50.0 MiB)
>         RX errors 0  dropped 0  overruns 0  frame 0
>         TX packets 165379  bytes 52434556 (50.0 MiB)
>         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
> 
> [root at mdserver ~]# 
> 
> 
> I've added "leftsourceip" but changed it to "rightsourceip" to both the ipsec initiator (hmserver) and the ipsec responder (core -
> ip-172-31-6-188) configuration.
>
>       [root at mdserver ~]# cat /etc/ipsec.d/amazoncore.conf 
> conn amazoncore
>         type=tunnel
>         authby=secret
>         auto=start
>         ike=aes256-sha1;modp1536,3des-md5;modp1024
>         forceencaps=yes
>         left=54.66.129.223
>         leftid=@blender
>         leftsourceip=10.1.0.1
>         leftsubnet=10.1.0.0/16
>         right=%defaultroute
>         rightid=@potatoe
>         rightsourceip=10.1.2.2
>         rightsubnet=10.1.2.0/24
> [root at mdserver ~]# 
> 
> 
> [root at ip-172-31-6-188 ~]# cat /etc/ipsec.d/forestlake.conf 
> conn forestlake
>         type=tunnel
>         authby=secret
>         auto=add
>         ike=aes256-sha1;modp1536,3des-md5;modp1024
>         forceencaps=yes
>         left=%defaultroute
>         leftid=@blender
>         leftsourceip=10.1.0.1
>         leftsubnet=10.1.0.0/16
>         right=%any
>         rightid=@potatoe
>         rightsourceip=10.1.2.2
>         rightsubnet=10.1.2.0/24
> [root at ip-172-31-6-188 ~]# 
> 
> 
> Based on your comments I've subsequently changed a few things which did not fix the problem and the same behaviour occurs, but
> might help get us closer.
> 
> On the ipsec initiator (hmserver), ipsec barf said:
> 
> + _________________________ ipsec_verify
> + ipsec verify --nocolour
> Verifying installed system and configuration files
> <SNIP>
> Two or more interfaces found, checking IP forwarding    [FAILED]
> [root at mdserver ~]# cat /proc/sys/net/ipv4/ip_forward 
> 0
> [root at mdserver ~]#
> 
> So I added "net.ipv4.ip_forward=1" to "/etc/sysctl.d/92-ipsec.conf" and ran "sysctl --system" so that:
>
>       [root at mdserver ~]# cat /proc/sys/net/ipv4/ip_forward 
> 1
> [root at mdserver ~]# 
> 
> 
> On the ipsec responder (core - ip-172-31-6-188), I changed the alias for 10.1.0.1 from the ethernet device to the lo device:
>
>       [root at ip-172-31-6-188 ~]# ip addr
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
>     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>     inet 127.0.0.1/8 scope host lo
>        valid_lft forever preferred_lft forever
>     inet6 ::1/128 scope host 
>        valid_lft forever preferred_lft forever
> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
>     link/ether 02:1b:09:ed:fb:c8 brd ff:ff:ff:ff:ff:ff
>     inet 172.31.6.188/20 brd 172.31.15.255 scope global dynamic eth0
>        valid_lft 2016sec preferred_lft 2016sec
>     inet 10.1.0.1/32 scope global eth0
>        valid_lft forever preferred_lft forever
>     inet6 fe80::1b:9ff:feed:fbc8/64 scope link 
>        valid_lft forever preferred_lft forever
> [root at ip-172-31-6-188 ~]# ip addr del 10.1.0.1/32 dev eth0
> [root at ip-172-31-6-188 ~]# ip addr add 10.1.0.1/32 dev lo
> [root at ip-172-31-6-188 ~]# ip addr
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
>     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>     inet 127.0.0.1/8 scope host lo
>        valid_lft forever preferred_lft forever
>     inet 10.1.0.1/32 scope global lo
>        valid_lft forever preferred_lft forever
>     inet6 ::1/128 scope host 
>        valid_lft forever preferred_lft forever
> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
>     link/ether 02:1b:09:ed:fb:c8 brd ff:ff:ff:ff:ff:ff
>     inet 172.31.6.188/20 brd 172.31.15.255 scope global dynamic eth0
>        valid_lft 1932sec preferred_lft 1932sec
>     inet6 fe80::1b:9ff:feed:fbc8/64 scope link 
>        valid_lft forever preferred_lft forever
> [root at ip-172-31-6-188 ~]# 
> 
> 
> 
> I also changed the configuration in the ipsec initiator to use the same left/right identies as the ipsec responder.
> 
> Based on your comment below in https://lists.libreswan.org/pipermail/swan/2015/001099.html, I also changed the "leftid" and
> "rightid" entries in all config and secrets files.
>
>       If using two libreswan installs, just set the ids using leftid=@something and rightid=@somethingelse
>
>        
>
>       That avoids using or defaulting to IPs being used as IDs, which is trick when NAT is involved (or when a remote
>       endpoint is on dynamic IP)
>
>        
>
>       Don't use leftid=@ipaddress, but use leftid=@somestring.
> 
> 
> After making all the above changes the problem persists.
> 
> 
> Cheers,
> 
> Paul MooreAstute Systems
> pmoore at astute-systems.com      0481 268 522       [btn_in_20x15.png] View my profile
> 
> On 20 March 2015 at 13:30, Paul Wouters <paul at nohats.ca> wrote:
>       On Fri, 20 Mar 2015, Paul Moore wrote:
> 
>
>             This is my first post to this list and I've been trying to figure out this problem for a few weeks
>             without asking for help because I thought I must be doing something stupid.
> 
>
>       to everyone: please never feel your question is too stupid. If something
>       is unclear after a google search, it is our problem in the
>       documentation. Please feel free to ask!
>
>             me to say "me too". Just like Dave, please also forgive me if I'm doing something wrong or breaking
>             mailing list etiquette.
> 
>
>       The only etiquette is to treat others as you like to be treated
>       yourself.
>
>             The basic problem is that a ping sent from the machine that initiated the tunnel (we'll call this the
>             ipsec initiatiator) and to the machine at the other end (we'll call this the ipsec responder) does not
>             work until a ping first comes from the ipsec responder back to the ipsec initiator. At that point, ping
>             responds only if the tunnel has had traffic pass through it in the last 30 seconds. Also, while the ping
>             from the ipsec initiator to the ipsec responder does not work, the ipsec initiatiator cannot even ping
>             itself.
> 
>
>       That's very odd. Did you observe if any of these pings were encrypted or
>       leaked in plaintext?
>
>             # ==== Output of mdserver command: "cat /etc/ipsec.d/*conf"
>             conn amazoncore
>                     type=tunnel
>                     authby=secret
>                     auto=start
>                     ike=aes256-sha1;modp1536,3des-md5;modp1024
>                     forceencaps=yes
>                     left=%defaultroute
>                     leftid=10.1.2.2
>                     leftsubnet=10.1.2.0/24
>                     right=54.66.129.223
>                     rightid=54.66.129.223
>                     rightsourceip=10.1.0.1
>                     rightsubnet=10.1.0.0/16
> 
>
>       Does the server have one interface in the 10.1.2.0/24 network? If so,
>       can you add leftsourceip=10.1.2.x to that? (where 10.1.2.x is the
>       IP the server has in the 10.1.2.0/24 network?)
>
>             conn forestlake
>                     type=tunnel
>                     authby=secret
>                     auto=add
>                     ike=aes256-sha1;modp1536,3des-md5;modp1024
>                     forceencaps=yes
>                     left=%defaultroute
>                     leftid=54.66.129.223
>                     leftsourceip=10.1.0.1
>                     leftsubnet=10.1.0.0/16
>                     right=%any
>                     rightid=10.1.2.2
>                     rightsubnet=10.1.2.0/24
> 
>
>       (note normally we don't flip "left" and "right" on both ends of a
>        connection. That is why we call it left/right and not source/dest :)
>
>       Also just to verify, I assume 10.1.0.1 is actually configured on the
>       amazon machine running libreswan, eg as alias on loopback or on an ethX device?
>       And not on another amazon located machine near the libreswan box?
>
>       to better understand what is happening, we would need to see the log
>       files produced by pluto. Running "ipsec barf" while libreswan is
>       running should give us the system config and the log files, so if you
>       can post that to a pastebin and give us the link, we could look into
>       this in more detail.
>
>       Paul
> 
> 
> 
> 
>


More information about the Swan mailing list