[Swan] Frequent dropped connections and martian source

[Alex]

Hi,

I have libreswan-3.27 on fedora29 on both ends with 5.0.10 that's been
running fine for a while. Over the last few days, the connection on
the local side has inexplicably disconnected from one of its two
net-to-net peers.

Just running "ipsec auto --up <tunnel-name>" on the local side usually
brings it up again. The remote side typically doesn't acknowledge that
the connection was lost, as it reports all tunnels are up. This has
happened about three times per day for the past week or so. I can't
think of anything that's changed with the system, and nothing has
changed with the configuration.

This time it didn't bring the connection up. This is reported in pluto.log:

May 21 20:14:21.606083: "orion-cyclops/1x1" #2019: initiate rekey of
IKEv2 CREATE_CHILD_SA IKE Rekey
May 21 20:14:21.607453: "orion-cyclops/1x1" #2028: message id
deadlock? wait sending, add to send next list using parent #2019
unacknowledged 2 next message id=2 ike exchange window 1
May 21 20:17:41.608603: "orion-cyclops/1x1" #2028: deleting state
(STATE_V2_REKEY_IKE_I0) and NOT sending notification

However, when I run "ipsec status", it appears to show the connection
is still active (or at least established):

000 #16: "orion-cyclops/1x1":500 STATE_V2_IPSEC_I (IPsec SA
established); EVENT_SA_REPLACE in 27684s; newest IPSEC; eroute owner;
isakmp#2; idle;
000 #16: "orion-cyclops/1x1" esp.552657e7 at 64.1.16.1
esp.5b8037ea at 68.195.193.42 tun.0 at 64.1.16.1 tun.0 at 68.195.193.42 ref=0
refhim=0 Traffic: ESPin=588B ESPout=588B! ESPmax=0B

I've also noticed martian source messages in the logs, but I don't
know if that's what's causing it, or that's the consequence of the
disconnected endpoint. The 192.168.1.0/24 is our local internal
network that's sometimes used to connect to networks behind the remote
network. I don't know where the 192.168.49.1 is coming from, as that's
not an IP or network we use.

[1376538.238061] IPv4: martian source 192.168.1.35 from 192.168.49.1,
on dev eth1
[1376538.238075] ll header: 00000000: ff ff ff ff ff ff 0c 47 c9 7b 4e b2 08 06
[1380207.332144] IPv4: martian source 192.168.1.105 from 192.168.49.1,
on dev eth1
[1380207.332159] ll header: 00000000: ff ff ff ff ff ff 0c 47 c9 7b 4e b2 08 06
[1393701.446458] IPv4: martian source 192.168.1.35 from 192.168.49.1,
on dev eth1

How do I troubleshoot this? The local side is a cable modem with a
static IP, but I don't think the connection is being dropped as we've
had no reports of that.

/etc/ipsec.conf:
config setup
        logfile=/var/log/pluto.log
        protostack=netkey
        hidetos=no
        klipsdebug=none
        keep_alive=60
include /etc/ipsec.d/*.conf

Local /etc/ipsec.d/orion-cyclops.conf (with domain name changed):
conn orion-cyclops
        ikev2=insist
        authby=rsasig
        auto=start
        dpddelay=10
        dpdtimeout=90
        dpdaction=clear
        rightid=@cyclops-orion
        rightsubnets={64.1.16.0/27,66.104.218.96/28,67.111.153.0/26}
        right=cyclops-dmz.example.com
        rightrsasigkey=0sAwEAAcauLbRx+x4jE...
        leftid=@orion-cyclops
        left=orion.example.com
        leftsubnets={192.168.1.0/24,192.168.6.0/24}
        leftrsasigkey=0sAwEAAeSMFxvoJaP54tr660X...