[Swan] libreswan/racoon interoperability problem with NAT-T

[Xinwei Hong]

Thank you Paul.

I added "compress=yes", seems not help. Compression_algorithm is only used
in phase 2, right? The failure here is phase 1. Firewall does not like the
issue either because packet on UDP 4500 can be seen on both end.

I have attached racoon log file. I see:

invalid length of payload

ERROR: phase1 negotiation failed due to time up.
8c6688cabaaf90ca:a8d8758b59948609

ERROR: phase2 negotiation failed due to time up waiting for phase1. ESP
10.2.128.240[0]->10.0.0.1[0]


Thanks,
Xinwei

On Fri, Mar 31, 2017 at 1:53 PM, Paul Wouters <paul at nohats.ca> wrote:

> On Thu, 30 Mar 2017, Xinwei Hong wrote:
>
> [moderator note: please next time post a pointer to a "pastebin" of the
> log instead of included large log files]
>
> I have a VPN setup between libreswan (pluto+netkey) and a racoon
>> (racoon+netkey), the racoon is behind a NAT device. The negotiation somehow
>> failed saying that "NAT-D payload #0 doesn't
>> match"
>>
>
> There is not a NAT payload error. Your device has more then one IP
> address, and it is trying to calculate the payload for both your
> IP addresses. It notes that 10.0.0.1 does not match. But it also
> notes that 10.2.128.240 works fine.
>
> Your actual problem is:
>
>         Mar 30 19:47:02 vvr-10 pluto[24271]: vvr-0:
>         "conn_vvr-0-ipsectunnel-0-remote-0" #1246: max number of
> retransmissions
>         (8) reached STATE_MAIN_I3.  Possible authentication failure: no
>         acceptable response to our first encrypted message
>
> Your racoon logs will likely have more information in it then your
> libreswan log, as racoon is the party rejecting the packet.
>
> conn conn_vvr-0-ipsectunnel-0
>>         authby=secret
>>         left=10.2.128.240
>>         right=10.2.128.241
>>         ike=3des-sha1;modp1024
>>         phase2alg=3des-sha1;modp1024
>>         leftsubnet=10.100.0.0/24
>>         rightsubnet=10.100.1.0/24
>>
>
> on racoon, we have racoon.conf
>> # Phase 1 (Main Mode) Configuration
>> remote 10.2.128.240 {
>>  exchange_mode main;
>>  proposal_check obey;
>>  lifetime time 28800 seconds;
>>  nat_traversal on;
>>  #script "phase1-up.sh" phase1_up;
>>  #script "phase1-down.sh" phase1_down;
>>  dpd_delay 15; dpd_retry 5; dpd_maxfail 5;
>>  proposal {
>>   encryption_algorithm 3des;
>>   hash_algorithm sha1;
>>   dh_group modp1024;
>>   authentication_method pre_shared_key;
>>  }
>> }
>>
>> # Phase 2 (Quick Mode) Configuration/Proposal (for IPsec SA).
>> sainfo anonymous {
>>  encryption_algorithm 3des;
>>  authentication_algorithm hmac_sha1;
>>  pfs_group modp1024;
>>  lifetime time 3600 seconds;
>>  compression_algorithm deflate;
>> }
>>
>
> The only mismatch I see is for compression. You can try and removing the
> compression line from racoon, or adding compress=yes to libreswan.
>
> Mar 30 19:47:52 testhost-601-1 racoon: ERROR: phase1 negotiation failed
>> due to time up. 80b77211a2f1ddba:141872152ca7772f
>>
>
> This is odd. Another possibility is that you have firewalled UDP 4500
> and so the NAT switching from UDP 500 to UDP 4500 is failing?
>
> Paul
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libreswan.org/pipermail/swan/attachments/20170331/81e671e7/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: racoon.log
Type: application/octet-stream
Size: 42003 bytes
Desc: not available
URL: <https://lists.libreswan.org/pipermail/swan/attachments/20170331/81e671e7/attachment-0001.obj>