[Swan-dev] overview of yesterday's test failures (please fix)

Paul Wouters paul at nohats.ca
Mon Oct 23 15:04:41 UTC 2017


On Tue, 17 Oct 2017, D. Hugh Redelmeier wrote:

> testing/pluto/ikev2-ddns-02 failed west:output-different
> 	script changed, reference output did not.

Fixed, the unbound.conf cp was wrong.

> testing/pluto/newoe-15-portpass failed road:output-different
> 	extra src policy

Passes for me and on testing.libreswan.org now

> testing/pluto/newoe-18-private-clear failed road:output-different
> 	extra src policy

Passes too.

> testing/pluto/newoe-18-poc-blockall failed road:output-different
> 	extra src policy

Same

> testing/pluto/newoe-18-private-clearall failed road:output-different
> 	extra src policy

Same

> testing/pluto/newoe-19-poc-poc-clear failed road:output-different
> 	extra src policy

same

> testing/pluto/newoe-20-ipv6 failed east:output-different road:output-different
> 	--- MASTER/testing/pluto/newoe-20-ipv6/road.console.txt
> 	+++ OUTPUT/testing/pluto/newoe-20-ipv6/road.console.txt
> 	@@ -11,8 +11,11 @@
> 	  echo "fe80::/10" >> /etc/ipsec.d/policies/clear
> 	 road #
> 	  cp /source/programs/configs/v6neighbor-hole.conf /etc/ipsec.d/
> 	+cp: cannot stat ‘/source/programs/configs/v6neighbor-hole.conf’: No such file or directory
> 	 road #
> 	  ipsec start
> 	+warning: could not open include filename: '/etc/ipsec.d/v6neighbor-hole.conf'
> 	+warning: could not open include filename: '/etc/ipsec.d/v6neighbor-hole.conf'
> 	 Redirecting to: systemctl start ipsec.service
> 	 road #
> 	  # ensure for tests acquires expire before our failureshunt=2m

the v6neighbor-hole.conf was fixed. but it still fails on policies. Not
yet sure what's going on.

> testing/pluto/newoe-21-liveness-clear failed east:output-different road:output-different
> 	road's script changed but reference log did not

liveness is difficult. too much noise. We need to redesign these.

> testing/pluto/certoe-07-nat-2-clients failed road:output-different
> 	extra src policy

Passes.

> testing/pluto/rawrsaoe-asymetric-nat failed east:output-different road:output-different
> 	some kind of real failure
>
> testing/pluto/dnsoe-01 failed east:output-different road:output-different
> 	some kind of real failure
>
> testing/pluto/dnsoe-02 failed east:output-different road:output-different
> 	some kind of real failure

These seem to be failures at the DNS level. I'm looking into that but
most likely the testcases need some tweaking.

> testing/pluto/dpd-01 failed west:output-different
> 	not sure.

This goes back to at least v3.15 or earlier. It seems related to dpd not
restarting a connection when dpdaction=%hold and rekey=yes. This goes
back to the discussion we have had in the past about what it means to
have a connection auto=add vs auto=start when we receive a DELETE or
when we receive a --up command.

> testing/pluto/ikev2-liveness-05 failed west:output-different
> 	script changed but not reference output

Similar to dpd-01, plus noise.

> testing/pluto/delete-sa-01 failed east:output-different west:output-different
> 	+whack error: SAwest-east unexpected argument "leftrsasigkey"
> 	etc.

this is due to Andrew's rewrite of the ipsec shell script. It has caused
failures in commands / options with a space in them. It does need
fixing.

> testing/pluto/nat-pluto-02-klips-klips failed road:output-different
> 	-006 #2: "road-eastnet-nat", type=ESP, add_time=1234567890, id='@east'
> 	+006 #2: "road-eastnet-nat", type=ESP, add_time=1234567890, inBytes=336, outBytes=336, id='@east'

Looks right, rerunning now and fixing...

> testing/pluto/xauth-pluto-17 failed road:output-different
> 	Worth examination, I think.

has been fixed.

> testing/pluto/xauth-pluto-25-mixed-addresspool failed north:output-different road:output-different
> 	looks bad:
> 	  ipsec whack --trafficstatus
> 	-006 #2: "north-east", username=xnorth, type=ESP, add_time=1234567890, inBytes=0, outBytes=0

passes now?

> testing/pluto/xauth-pluto-25-lsw299 failed north:output-different road:output-different
> 	looks bad:
> 	  ipsec whack --trafficstatus
> 	-006 #2: "road-east", username=xroad, type=ESP, add_time=1234567890, inBytes=336, outBytes=336

passes now.

> testing/pluto/netkey-klips-pluto-03 failed west:output-different
> 	lots of differences in xfrm policy
>
> testing/pluto/klips-netkey-pluto-06 failed west:output-different
> 	lots of differences in xfrm policy

I have noticed these. I suspect a sanitizer on xfrm causes this. But
need to investigate further.

> testing/pluto/interop-ikev2-strongswan-13-ah-initiator failed west:output-different
> 	--- MASTER/testing/pluto/interop-ikev2-strongswan-13-ah-initiator/west.console.txt
> 	+++ OUTPUT/testing/pluto/interop-ikev2-strongswan-13-ah-initiator/west.console.txt
> 	@@ -39,10 +39,9 @@
> 	 sending packet: from 192.1.2.45[500] to 192.1.2.23[500] (XXX bytes)
> 	 received packet: from 192.1.2.23[500] to 192.1.2.45[500] (XXX bytes)
> 	 parsed IKE_SA_INIT response 0 [ SA KE No N(FRAG_SUP) N(NATD_S_IP) N(NATD_D_IP) ]
> 	-sending cert request for "C=CA, ST=Ontario, L=Toronto, O=Libreswan, OU=Test Department, CN=Libreswan test CA for mainca, E=testing at libreswan.org"

This bug was found. One test case copied in the EC CA cert and didnt
remove it. So other tests would then add sending the CERTREQ for it.
Depending on if the ec test was run or it was a clean kvm, the test
would cause all other tests to be thrown off. swan-prep now properly
cleans up the dir so this flipflopping no longer happens.

> testing/pluto/interop-ikev2-strongswan-35-ipsec-rekey failed west:output-different
> 	--- MASTER/testing/pluto/interop-ikev2-strongswan-35-ipsec-rekey/west.console.txt
> 	+++ OUTPUT/testing/pluto/interop-ikev2-strongswan-35-ipsec-rekey/west.console.txt
> 	@@ -87,8 +87,10 @@
> 	  strongswan status
> 	 Security Associations (1 up, 0 connecting):
> 	 westnet-eastnet-ikev2[1]: ESTABLISHED XXX second ago, 192.1.2.45[west]...192.1.2.23[east]
> 	-westnet-eastnet-ikev2{6}:  INSTALLED, TUNNEL, reqid 1, ESP SPIs: SPISPI_i SPISPI_o
> 	+westnet-eastnet-ikev2{6}:  DELETING, TUNNEL, reqid 1
> 	 westnet-eastnet-ikev2{6}:   192.0.1.0/24 === 192.0.2.0/24
> 	+westnet-eastnet-ikev2{7}:  INSTALLED, TUNNEL, reqid 1, ESP SPIs: SPISPI_i SPISPI_o
> 	+westnet-eastnet-ikev2{7}:   192.0.1.0/24 === 192.0.2.0/24
> 	 west #
> 	  echo done
> 	 done

I've tried adding a sleep here to prevent a race condition in the
deleting. It seems to have helped?

> testing/pluto/interop-ikev2-strongswan-35-rekey-reauth failed east:output-different west:output-different
> 	reqid changed

Not only that, the change I am confused about is "erouted" vs "prospective erouted"

I haven't fixed it because I'm not sure what's happening here. Likely an
older change that never got updated.

> testing/pluto/interop-ikev2-strongswan-35-responder-rekey-pfs failed west:output-different
> 	--- MASTER/testing/pluto/interop-ikev2-strongswan-35-responder-rekey-pfs/west.console.txt
> 	+++ OUTPUT/testing/pluto/interop-ikev2-strongswan-35-responder-rekey-pfs/west.console.txt
> 	@@ -36,10 +36,8 @@
> 	 westnet-eastnet-ikev2[1]: ESTABLISHED XXX second ago, 192.1.2.45[west]...192.1.2.23[east]
> 	 westnet-eastnet-ikev2{1}:  DELETING, TUNNEL, reqid 1
> 	 westnet-eastnet-ikev2{1}:   192.0.1.0/24 === 192.0.2.0/24
> 	-westnet-eastnet-ikev2{2}:  DELETING, TUNNEL, reqid 1
> 	+westnet-eastnet-ikev2{2}:  REKEYING, TUNNEL, reqid 1, expires in 59 minutes
> 	 westnet-eastnet-ikev2{2}:   192.0.1.0/24 === 192.0.2.0/24
> 	-westnet-eastnet-ikev2{3}:  INSTALLED, TUNNEL, reqid 1, ESP SPIs: SPISPI_i SPISPI_o
> 	-westnet-eastnet-ikev2{3}:   192.0.1.0/24 === 192.0.2.0/24
> 	 west #
> 	  echo done
> 	 done

same race condition as above.

> testing/pluto/dnssec-pluto-01 failed west:output-different
> 	--- MASTER/testing/pluto/dnssec-pluto-01/west.console.txt
> 	+++ OUTPUT/testing/pluto/dnssec-pluto-01/west.console.txt
> 	@@ -39,8 +39,6 @@
> 	  ipsec auto --status | egrep "oriented|east-from-hosts"
> 	 000 "westnet-eastnet-etc-hosts": 192.0.1.0/24===192.1.2.45<192.1.2.45>[@west]...192.1.2.23<east-from-hosts-file>[@east]===192.0.2.0/24; unrouted; eroute owner: #0
> 	 000 "westnet-eastnet-etc-hosts":     oriented; my_ip=unset; their_ip=unset; my_updown=ipsec _updown;
> 	-000 "westnet-eastnet-etc-hosts-auto-add": 192.0.1.0/24===192.1.2.45<192.1.2.45>[@west]...192.1.2.23<east-from-hosts-file>[@east]===192.0.2.0/24; unrouted; eroute owner: #0
> 	-000 "westnet-eastnet-etc-hosts-auto-add":     oriented; my_ip=unset; their_ip=unset; my_updown=ipsec _updown;
> 	 west #
> 	  echo "initdone"
> 	 initdone

This is a bug that must be fixed still. Those entries from /etc/hosts
should have loaded fine but did not.

> testing/pluto/ikev2-55-ipseckey-01 passed
> testing/pluto/ikev2-55-ipseckey-02 failed road:output-different
> 	--- MASTER/testing/pluto/ikev2-55-ipseckey-02/road.console.txt
> 	+++ OUTPUT/testing/pluto/ikev2-55-ipseckey-02/road.console.txt
> 	@@ -83,9 +83,9 @@
> 	 133 "road-east-2" #1: STATE_PARENT_I1: initiate
> 	 133 "road-east-2" #1: STATE_PARENT_I1: sent v2I1, expected v2R1
> 	 002 "road-east-2" #1: suppressing retransmit because IMPAIR_RETRANSMITS is set.
> 	-003 "road-east-2" #1: Can't find the private key from the NSS CKA_ID
> 	-003 "road-east-2" #1: Failed to find our RSA key
> 	-000 "road-east-2" #1: realse whack for IKE SA, but releasing whack for pending IPSEC SA
> 	+003 "road-east-2" #1: Can't find the certificate or private key from the NSS CKA_ID
> 	+003 "road-east-2" #1: DigSig: failed to find our RSA key
> 	+000 "road-east-2" #1: release whack for IKE SA, but releasing whack for pending IPSEC SA
> 	 road #
> 	  ping -n -c 4 -I 192.1.3.209 192.1.2.23
> 	 PING 192.1.2.23 (192.1.2.23) from 192.1.3.209 : 56(84) bytes of data.

Fixed. the error message changed due to Digital Signatures support.

> testing/pluto/nss-cert-crl-03-strict failed west:output-different
> 	--- MASTER/testing/pluto/nss-cert-crl-03-strict/west.console.txt
> 	+++ OUTPUT/testing/pluto/nss-cert-crl-03-strict/west.console.txt

passes for us?

>
> testing/pluto/nss-cert-nosecret failed west:output-different

also passes for us?

> testing/pluto/nss-cert-09-notyetvalid-initiator failed east:output-different west:output-different
> 	--- MASTER/testing/pluto/nss-cert-09-notyetvalid-initiator/east.console.txt
> 	+++ OUTPUT/testing/pluto/nss-cert-09-notyetvalid-initiator/east.console.txt
> 	@@ -17,7 +17,6 @@
> 	  # will only show up on east - note "expired" is wrong and should be "not yet valid"
> 	 east #
> 	  grep "ERROR" /tmp/pluto.log
> 	-"nss-cert" #1: ERROR: Peer's Certificate has expired.

These tend to be due to libfaketime use. It seems to not work for
everyone and/or people run without regenerating the certs/keys for
over 2 weeks. All the notyetvalid tests will show that in this case.

> testing/pluto/ipsec-hostkey-ckaid-02-fips failed west:output-different
> 	--- MASTER/testing/pluto/ipsec-hostkey-ckaid-02-fips/west.console.txt
> 	+++ OUTPUT/testing/pluto/ipsec-hostkey-ckaid-02-fips/west.console.txt
> 	@@ -4,14 +4,18 @@
> 	 FIPS mode enabled.
> 	 west #
> 	  ipsec newhostkey
> 	-Generated RSA key pair with CKAID <<CKAID#1>> was stored in the NSS database
> 	+FIPS HMAC integrity verification test failed.

I have these too, I suspect Andrew made an error on the .hmac file here?
Or this really only works when installed in /usr and not /usr/local ?

Paul


More information about the Swan-dev mailing list