[Swan] Traffic _stops_ routing down tunnel

Paul Moore pmoore at astute-systems.com
Fri Apr 3 05:44:16 EEST 2015


Hi Paul,

Do you have any insight on this one?

I've been playing with accept_redirects, send_redirects, secure_redirects
and rp_filter and have turned everything off, but it doesn't change
anything.

I'm limping along for now by sending syslog traffic from the AWS host to
the other end to keep the ability of the other end to ping the AWS host,
but it's not really regular enough to be reliable.

Cheers,

Paul Moore
Astute Systems
pmoore at astute-systems.com      0481 268 522       View my profile
<http://www.linkedin.com/profile/view?id=465982>

On 20 March 2015 at 15:57, Paul Moore <pmoore at astute-systems.com> wrote:

> Hi Paul,
>
> Thanks for your reply.
>
> Before changing any configuration, I've pasted the output of the ipsec
> initiator at http://pastebin.com/ct532Ewc and the ipsec responder at
> http://pastebin.com/1bzGcq1d
>
> Does the server have one interface in the 10.1.2.0/24 network? If so,
>> can you add leftsourceip=10.1.2.x to that? (where 10.1.2.x is the
>> IP the server has in the 10.1.2.0/24 network?)
>
>
> The ipsec initiator (mdserver) has it's only ethernet address as
> 10.1.2.2/24 on the internal network.
>
> *[root at mdserver ~]# ifconfig*
> *enp0s29f0u2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500*
> *        ether 02:21:5e:0a:a9:1f  txqueuelen 1000  (Ethernet)*
> *        RX packets 39263  bytes 2555803 (2.4 MiB)*
> *        RX errors 0  dropped 0  overruns 0  frame 0*
> *        TX packets 0  bytes 0 (0.0 B)*
> *        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0*
>
> *enp11s0f0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500*
> *        inet 10.1.2.2  netmask 255.255.255.0  broadcast 10.1.2.255*
> *        inet6 fe80::221:5eff:fe09:a91c  prefixlen 64  scopeid 0x20<link>*
> *        ether 00:21:5e:09:a9:1c  txqueuelen 1000  (Ethernet)*
> *        RX packets 184092  bytes 41376693 (39.4 MiB)*
> *        RX errors 0  dropped 29  overruns 0  frame 0*
> *        TX packets 138809 <138809>  bytes 43058412 (41.0 MiB)*
> *        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0*
>
> *enp11s0f1: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500*
> *        ether 00:21:5e:09:a9:1e  txqueuelen 1000  (Ethernet)*
> *        RX packets 0  bytes 0 (0.0 B)*
> *        RX errors 0  dropped 0  overruns 0  frame 0*
> *        TX packets 0  bytes 0 (0.0 B)*
> *        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0*
>
> *lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536*
> *        inet 127.0.0.1  netmask 255.0.0.0*
> *        inet6 ::1  prefixlen 128  scopeid 0x10<host>*
> *        loop  txqueuelen 0  (Local Loopback)*
> *        RX packets 165379 <165379>  bytes 52434556 (50.0 MiB)*
> *        RX errors 0  dropped 0  overruns 0  frame 0*
> *        TX packets 165379 <165379>  bytes 52434556 (50.0 MiB)*
> *        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0*
>
> *[root at mdserver ~]# *
>
>
> I've added "leftsourceip" but changed it to "rightsourceip" to both the
> ipsec initiator (hmserver) and the ipsec responder (core - ip-172-31-6-188)
> configuration.
>
> *[root at mdserver ~]# cat /etc/ipsec.d/amazoncore.conf *
> *conn amazoncore*
> *        type=tunnel*
> *        authby=secret*
> *        auto=start*
> *        ike=aes256-sha1;modp1536,3des-md5;modp1024*
> *        forceencaps=yes*
> *        left=54.66.129.223*
> *        leftid=@blender*
> *        leftsourceip=10.1.0.1*
> *        leftsubnet=10.1.0.0/16 <http://10.1.0.0/16>*
> *        right=%defaultroute*
> *        rightid=@potatoe*
> *        rightsourceip=10.1.2.2*
> *        rightsubnet=10.1.2.0/24 <http://10.1.2.0/24>*
> *[root at mdserver ~]# *
>
>
> *[root at ip-172-31-6-188 ~]# cat /etc/ipsec.d/forestlake.conf *
> *conn forestlake*
> *        type=tunnel*
> *        authby=secret*
> *        auto=add*
> *        ike=aes256-sha1;modp1536,3des-md5;modp1024*
> *        forceencaps=yes*
> *        left=%defaultroute*
> *        leftid=@blender*
> *        leftsourceip=10.1.0.1*
> *        leftsubnet=10.1.0.0/16 <http://10.1.0.0/16>*
> *        right=%any*
> *        rightid=@potatoe*
> *        rightsourceip=10.1.2.2*
> *        rightsubnet=10.1.2.0/24 <http://10.1.2.0/24>*
> *[root at ip-172-31-6-188 ~]# *
>
>
> Based on your comments I've subsequently changed a few things which did
> not fix the problem and the same behaviour occurs, but might help get us
> closer.
>
> On the ipsec initiator (hmserver), ipsec barf said:
>
>
> *+ _________________________ ipsec_verify*
> *+ ipsec verify --nocolour*
> *Verifying installed system and configuration files*
> *<SNIP>*
> *Two or more interfaces found, checking IP forwarding    [FAILED]*
> *[root at mdserver ~]# cat /proc/sys/net/ipv4/ip_forward *
> *0*
> *[root at mdserver ~]#*
>
> So I added "net.ipv4.ip_forward=1" to "/etc/sysctl.d/92-ipsec.conf" and
> ran "sysctl --system" so that:
>
> *[root at mdserver ~]# cat /proc/sys/net/ipv4/ip_forward *
> *1*
> *[root at mdserver ~]# *
>
>
> On the ipsec responder (core - ip-172-31-6-188), I changed the alias for
> 10.1.0.1 from the ethernet device to the lo device:
>
> *[root at ip-172-31-6-188 ~]# ip addr*
> *1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN *
> *    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00*
> *    inet 127.0.0.1/8 <http://127.0.0.1/8> scope host lo*
> *       valid_lft forever preferred_lft forever*
> *    inet6 ::1/128 scope host *
> *       valid_lft forever preferred_lft forever*
> *2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
> state UP qlen 1000*
> *    link/ether 02:1b:09:ed:fb:c8 brd ff:ff:ff:ff:ff:ff*
> *    inet 172.31.6.188/20 <http://172.31.6.188/20> brd 172.31.15.255 scope
> global dynamic eth0*
> *       valid_lft 2016sec preferred_lft 2016sec*
> *    inet 10.1.0.1/32 <http://10.1.0.1/32> scope global eth0*
> *       valid_lft forever preferred_lft forever*
> *    inet6 fe80::1b:9ff:feed:fbc8/64 scope link *
> *       valid_lft forever preferred_lft forever*
> *[root at ip-172-31-6-188 ~]# ip addr del 10.1.0.1/32 <http://10.1.0.1/32>
> dev eth0*
> *[root at ip-172-31-6-188 ~]# ip addr add 10.1.0.1/32 <http://10.1.0.1/32>
> dev lo*
> *[root at ip-172-31-6-188 ~]# ip addr*
> *1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN *
> *    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00*
> *    inet 127.0.0.1/8 <http://127.0.0.1/8> scope host lo*
> *       valid_lft forever preferred_lft forever*
> *    inet 10.1.0.1/32 <http://10.1.0.1/32> scope global lo*
> *       valid_lft forever preferred_lft forever*
> *    inet6 ::1/128 scope host *
> *       valid_lft forever preferred_lft forever*
> *2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
> state UP qlen 1000*
> *    link/ether 02:1b:09:ed:fb:c8 brd ff:ff:ff:ff:ff:ff*
> *    inet 172.31.6.188/20 <http://172.31.6.188/20> brd 172.31.15.255 scope
> global dynamic eth0*
> *       valid_lft 1932sec preferred_lft 1932sec*
> *    inet6 fe80::1b:9ff:feed:fbc8/64 scope link *
> *       valid_lft forever preferred_lft forever*
> *[root at ip-172-31-6-188 ~]# *
>
>
>
> I also changed the configuration in the ipsec initiator to use the same
> left/right identies as the ipsec responder.
>
> Based on your comment below in
> https://lists.libreswan.org/pipermail/swan/2015/001099.html, I also
> changed the "leftid" and "rightid" entries in all config and secrets files.
>
>
> If using two libreswan installs, just set the ids using leftid=@something
>> and rightid=@somethingelse
>
>
>
> That avoids using or defaulting to IPs being used as IDs, which is trick
>> when NAT is involved (or when a remote endpoint is on dynamic IP)
>
>
>
> Don't use leftid=@ipaddress, but use leftid=@somestring.
>
>
> After making all the above changes the problem persists.
>
>
> Cheers,
>
> Paul Moore
> Astute Systems
> pmoore at astute-systems.com      0481 268 522       View my profile
> <http://www.linkedin.com/profile/view?id=465982>
>
> On 20 March 2015 at 13:30, Paul Wouters <paul at nohats.ca> wrote:
>
>> On Fri, 20 Mar 2015, Paul Moore wrote:
>>
>>
>>> This is my first post to this list and I've been trying to figure out
>>> this problem for a few weeks
>>> without asking for help because I thought I must be doing something
>>> stupid.
>>>
>>
>> to everyone: please never feel your question is too stupid. If something
>> is unclear after a google search, it is our problem in the
>> documentation. Please feel free to ask!
>>
>>  me to say "me too". Just like Dave, please also forgive me if I'm doing
>>> something wrong or breaking
>>> mailing list etiquette.
>>>
>>
>> The only etiquette is to treat others as you like to be treated
>> yourself.
>>
>>  The basic problem is that a ping sent from the machine that initiated
>>> the tunnel (we'll call this the
>>> ipsec initiatiator) and to the machine at the other end (we'll call this
>>> the ipsec responder) does not
>>> work until a ping first comes from the ipsec responder back to the ipsec
>>> initiator. At that point, ping
>>> responds only if the tunnel has had traffic pass through it in the last
>>> 30 seconds. Also, while the ping
>>> from the ipsec initiator to the ipsec responder does not work, the ipsec
>>> initiatiator cannot even ping
>>> itself.
>>>
>>
>> That's very odd. Did you observe if any of these pings were encrypted or
>> leaked in plaintext?
>>
>>  # ==== Output of mdserver command: "cat /etc/ipsec.d/*conf"
>>> conn amazoncore
>>>         type=tunnel
>>>         authby=secret
>>>         auto=start
>>>         ike=aes256-sha1;modp1536,3des-md5;modp1024
>>>         forceencaps=yes
>>>         left=%defaultroute
>>>         leftid=10.1.2.2
>>>         leftsubnet=10.1.2.0/24
>>>         right=54.66.129.223
>>>         rightid=54.66.129.223
>>>         rightsourceip=10.1.0.1
>>>         rightsubnet=10.1.0.0/16
>>>
>>
>> Does the server have one interface in the 10.1.2.0/24 network? If so,
>> can you add leftsourceip=10.1.2.x to that? (where 10.1.2.x is the
>> IP the server has in the 10.1.2.0/24 network?)
>>
>>  conn forestlake
>>>         type=tunnel
>>>         authby=secret
>>>         auto=add
>>>         ike=aes256-sha1;modp1536,3des-md5;modp1024
>>>         forceencaps=yes
>>>         left=%defaultroute
>>>         leftid=54.66.129.223
>>>         leftsourceip=10.1.0.1
>>>         leftsubnet=10.1.0.0/16
>>>         right=%any
>>>         rightid=10.1.2.2
>>>         rightsubnet=10.1.2.0/24
>>>
>>
>> (note normally we don't flip "left" and "right" on both ends of a
>>  connection. That is why we call it left/right and not source/dest :)
>>
>> Also just to verify, I assume 10.1.0.1 is actually configured on the
>> amazon machine running libreswan, eg as alias on loopback or on an ethX
>> device?
>> And not on another amazon located machine near the libreswan box?
>>
>> to better understand what is happening, we would need to see the log
>> files produced by pluto. Running "ipsec barf" while libreswan is
>> running should give us the system config and the log files, so if you
>> can post that to a pastebin and give us the link, we could look into
>> this in more detail.
>>
>> Paul
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libreswan.org/pipermail/swan/attachments/20150403/e5afe713/attachment-0001.html>


More information about the Swan mailing list