[Swan-dev] more analysis of failing tests
D. Hugh Redelmeier
hugh at mimosa.com
Sun Oct 22 17:15:30 UTC 2017
I ran the tests yesterday, then Paul fixed some of them, and then I
reran the failures.
Real progress, but still work to be done. At least some should be
easy.
I've started to point fingers in a few cases. I hope that this gets
attention for these problems
testing/pluto/ikev2-delete-06-start-both failed west:output-different
state numbers differed
lost ping packet
[FIXED]
testing/pluto/ikev2-62-host-ondemand failed north:output-different
-4 packets transmitted, 2 received, 50% packet loss, time XXXX
+5 packets transmitted, 3 received, 40% packet loss, time XXXX
ipsec whack --trafficstatus
-006 #2: "north-east", type=ESP, add_time=1234567890, inBytes=168, outBytes=168, id='@east'
+006 #3: "north-east", type=ESP, add_time=1234567890, inBytes=0, outBytes=252, id='@east'
+006 #4: "north-east", type=ESP, add_time=1234567890, inBytes=252, outBytes=0, id='@east'
testing/pluto/ikev2-ddns-02 failed west:output-different
script doesn't match reference output
[FIXED]
testing/pluto/ikev1-algo-esp-sha2-01-netkey-klips failed west:output-different
one packet lost
[OK]
testing/pluto/ikev1-algo-esp-sha2-02-netkey-klips failed west:output-different
one packet lost
[OK]
testing/pluto/nflog-01-global failed west:output-different
only difference is in order of log entries
testing/pluto/newoe-15-portpass failed road:output-different
+src 192.1.3.209 dst 192.1.2.23
+ proto esp spi 0xSPISPIXX reqid REQID mode transport
+ replay-window 0
+ sel src 192.1.3.209/32 dst 192.1.2.23/32 proto icmp type 8 code 0 dev eth0
testing/pluto/newoe-18-private-clear failed road:output-different
XFRM state:
+src 192.1.3.209 dst 192.1.2.23
+ proto esp spi 0xSPISPIXX reqid REQID mode transport
+ replay-window 0
+ sel src 192.1.3.209/32 dst 192.1.2.23/32 proto icmp type 8 code 0 dev eth0
testing/pluto/newoe-18-poc-blockall failed road:output-different
XFRM state:
+src 192.1.3.209 dst 192.1.2.23
+ proto esp spi 0xSPISPIXX reqid REQID mode transport
+ replay-window 0
+ sel src 192.1.3.209/32 dst 192.1.2.23/32 proto icmp type 8 code 0 dev eth0
testing/pluto/newoe-18-private-clearall failed road:output-different
XFRM state:
+src 192.1.3.209 dst 192.1.2.23
+ proto esp spi 0xSPISPIXX reqid REQID mode transport
+ replay-window 0
+ sel src 192.1.3.209/32 dst 192.1.2.23/32 proto icmp type 8 code 0 dev eth0
testing/pluto/newoe-19-poc-poc-clear failed road:output-different
+src 192.1.3.209 dst 192.1.2.23
+ proto esp spi 0xSPISPIXX reqid REQID mode transport
+ replay-window 0
+ sel src 192.1.3.209/32 dst 192.1.2.23/32 proto icmp type 8 code 0 dev eth0
testing/pluto/newoe-21-liveness-clear failed east:output-different road:output-different
east:
ipsec whack --trafficstatus
-006 #4: "private-or-clear#192.1.3.0/24"[2] ...192.1.3.209, type=ESP, add_time=1234567890, inBytes=0, outBytes=0, id='ID_NULL'
road:
- ping -w 6 -n -c 1 -I 192.1.3.209 192.1.2.23
+ ping -n -c 1 -I 192.1.3.209 192.1.2.23
script changed two years ago [Paul]
testing/pluto/certoe-04-poc-packet failed east:output-different nic:output-different road:output-different
road #
- echo "192.1.2.0/24" >> /etc/ipsec.d/policies/private-or-clear
+ echo "192.1.2.23/32" >> /etc/ipsec.d/policies/private-or-clear
lots of evidence that script does not match reference output
[FIXED]
testing/pluto/certoe-07-nat-2-clients failed road:output-different
+src 192.1.3.209 dst 192.1.2.23
+ proto esp spi 0xSPISPIXX reqid REQID mode transport
+ replay-window 0
+ sel src 192.1.3.209/32 dst 192.1.2.23/32 proto icmp type 8 code 0 dev eth0
testing/pluto/certoe-09-packet-host failed road:output-different
--- 192.1.2.23 ping statistics ---
-1 packets transmitted, 0 received, 100% packet loss, time XXXX
+2 packets transmitted, 0 received, 100% packet loss, time XXXX
etc.
testing/pluto/certoe-09-packet-host-2 failed east:output-different nic:output-different road:output-different
Paul is fixing this.
[FIXED]
testing/pluto/rawrsaoe-asymetric-nat failed east:output-different road:output-different
east: trailing blanks on reference output
road:
002 "private-or-clear#192.1.2.0/24"[1] ...192.1.2.23 #2: suppressing retransmit because IMPAIR_RETRANSMITS is set.
-002 "private-or-clear#192.1.2.0/24"[1] ...192.1.2.23 #2: received INTERNAL_IP4_ADDRESS 10.0.10.1
-002 "private-or-clear#192.1.2.0/24"[1] 10.0.10.1/32=== ...192.1.2.23 #2: negotiated connection [10.0.10.1-10.0.10.1:0-65535 0] -> [192.1.2.23-192.1.2.23:0-65535 0]
and more
testing/pluto/dnsoe-01 failed east:output-different road:output-different
missing connections
testing/pluto/dnsoe-02 failed east:output-different road:output-different
missing connections
testing/pluto/dpd-01 failed west:output-different
not clear to me
testing/pluto/ikev2-liveness-05 failed west:output-different
script does not match reference output [Antony]
testing/pluto/ikev2-liveness-09 failed west:output-different
packet dropped
[OK]
testing/pluto/delete-sa-01 failed east:output-different west:output-different
script does not match reference output [Paul]
testing/pluto/x509-pluto-frag-01 failed road:output-different
complex retransmission
testing/pluto/nat-pluto-02-klips-klips failed road:output-different
ipsec whack --trafficstatus
-006 #2: "road-eastnet-nat", type=ESP, add_time=1234567890, id='@east'
+006 #2: "road-eastnet-nat", type=ESP, add_time=1234567890, inBytes=252, outBytes=336, id='@east'
testing/pluto/nat-pluto-04 failed east:output-different
Huh?
arp -an
-? (192.1.2.254) at 12:00:00:de:ad:ba [ether] on eth1
testing/pluto/xauth-pluto-04 failed road:output-different
run used Main Mode, reference used Aggressive Mode [Paul]
[FIXED]
testing/pluto/xauth-pluto-23 failed east:output-different north:output-different
east: serial numbers changed
north: scripts conflict with logs [Antony]
testing/pluto/xauth-pluto-24-static-addresspool failed road:output-different
--- 192.0.2.254 ping statistics ---
-4 packets transmitted, 0 received, 100% packet loss, time XXXX
+0 packets transmitted, 0 received
[FIXED]
testing/pluto/klips-algo-twofish-01 failed west:output-different
one packet lost
testing/pluto/klips-algo-cast-01 failed west:output-different
one packet lost
testing/pluto/ah-pluto-07-klips-netkey failed west:output-different
one packet lost
testing/pluto/netkey-klips-pluto-03 failed west:output-different
XFRM state:
src 192.1.2.23 dst 192.1.2.45
proto esp spi 0xSPISPIXX reqid REQID mode tunnel
- replay-window 32 flag af-unspec
- auth-trunc hmac(sha1) 0xHASHKEY 96
- enc cbc(aes) 0xENCKEY
-src 192.1.2.45 dst 192.1.2.23
- proto esp spi 0xSPISPIXX reqid REQID mode tunnel
- replay-window 32 flag af-unspec
- auth-trunc hmac(sha1) 0xHASHKEY 96
- enc cbc(aes) 0xENCKEY
+ replay-window 0
+ sel src 192.1.2.23/32 dst 192.1.2.45/32
XFRM policy:
src 192.0.1.0/24 dst 192.0.2.0/24
dir out priority 2344 ptype main
- tmpl src 192.1.2.45 dst 192.1.2.23
- proto esp reqid REQID mode tunnel
+ tmpl src 0.0.0.0 dst 0.0.0.0
+ proto esp reqid REQID mode transport
src 192.0.1.0/24 dst 192.0.2.0/24 proto icmp
dir out priority 1768 ptype main
-src 192.0.2.0/24 dst 192.0.1.0/24
- dir fwd priority 2344 ptype main
- tmpl src 192.1.2.23 dst 192.1.2.45
- proto esp reqid REQID mode tunnel
-src 192.0.2.0/24 dst 192.0.1.0/24
- dir in priority 2344 ptype main
- tmpl src 192.1.2.23 dst 192.1.2.45
- proto esp reqid REQID mode tunnel
src 192.0.2.0/24 dst 192.0.1.0/24 proto icmp
dir fwd priority 1768 ptype main
src 192.0.2.0/24 dst 192.0.1.0/24 proto icmp
testing/pluto/klips-netkey-pluto-06 failed west:output-different
--- MASTER/testing/pluto/klips-netkey-pluto-06/west.console.txt
+++ OUTPUT/testing/pluto/klips-netkey-pluto-06/west.console.txt
@@ -68,29 +68,15 @@
XFRM state:
src 192.1.2.23 dst 192.1.2.45
proto esp spi 0xSPISPIXX reqid REQID mode tunnel
- replay-window 32 flag af-unspec
- auth-trunc hmac(sha1) 0xHASHKEY 96
- enc cbc(aes) 0xENCKEY
-src 192.1.2.45 dst 192.1.2.23
- proto esp spi 0xSPISPIXX reqid REQID mode tunnel
- replay-window 32 flag af-unspec
- auth-trunc hmac(sha1) 0xHASHKEY 96
- enc cbc(aes) 0xENCKEY
+ replay-window 0
+ sel src 192.1.2.23/32 dst 192.1.2.45/32
XFRM policy:
src 192.0.1.0/24 dst 192.0.2.0/24
dir out priority 2344 ptype main
- tmpl src 192.1.2.45 dst 192.1.2.23
- proto esp reqid REQID mode tunnel
+ tmpl src 0.0.0.0 dst 0.0.0.0
+ proto esp reqid REQID mode transport
src 192.0.1.0/24 dst 192.0.2.0/24 proto icmp
dir out priority 1768 ptype main
-src 192.0.2.0/24 dst 192.0.1.0/24
- dir fwd priority 2344 ptype main
- tmpl src 192.1.2.23 dst 192.1.2.45
- proto esp reqid REQID mode tunnel
-src 192.0.2.0/24 dst 192.0.1.0/24
- dir in priority 2344 ptype main
- tmpl src 192.1.2.23 dst 192.1.2.45
- proto esp reqid REQID mode tunnel
src 192.0.2.0/24 dst 192.0.1.0/24 proto icmp
dir fwd priority 1768 ptype main
src 192.0.2.0/24 dst 192.0.1.0/24 proto icmp
testing/pluto/interop-ikev2-strongswan-04-x509-responder failed west:output-different
one extra retransmission (out of 4!)
testing/pluto/interop-ikev2-strongswan-07-strongswan failed west:output-different
CHILD_SA westnet-eastnet-ikev2{1} established with SPIs SPISPI_i SPISPI_o and TS 192.0.1.0/24 === 192.0.2.0/24
+received AUTH_LIFETIME of XXXXs, scheduling reauthentication in XXXXs
+peer supports MOBIKE
connection 'westnet-eastnet-ikev2' established successfully
testing/pluto/interop-ikev2-strongswan-13-ah-initiator failed west:output-different
--- MASTER/testing/pluto/interop-ikev2-strongswan-13-ah-initiator/west.console.txt
+++ OUTPUT/testing/pluto/interop-ikev2-strongswan-13-ah-initiator/west.console.txt
@@ -39,10 +39,9 @@
sending packet: from 192.1.2.45[500] to 192.1.2.23[500] (XXX bytes)
received packet: from 192.1.2.23[500] to 192.1.2.45[500] (XXX bytes)
parsed IKE_SA_INIT response 0 [ SA KE No N(FRAG_SUP) N(NATD_S_IP) N(NATD_D_IP) ]
-sending cert request for "C=CA, ST=Ontario, L=Toronto, O=Libreswan, OU=Test Department, CN=Libreswan test CA for mainca, E=testing at libreswan.org"
authentication of 'west' (myself) with pre-shared key
establishing CHILD_SA westnet-eastnet-ikev2{1}
-generating IKE_AUTH request 1 [ IDi N(INIT_CONTACT) CERTREQ IDr AUTH SA TSi TSr N(MOBIKE_SUP) N(ADD_4_ADDR) N(ADD_4_ADDR) N(EAP_ONLY) N(MSG_ID_SYN_SUP) ]
+generating IKE_AUTH request 1 [ IDi N(INIT_CONTACT) IDr AUTH SA TSi TSr N(MOBIKE_SUP) N(ADD_4_ADDR) N(ADD_4_ADDR) N(EAP_ONLY) N(MSG_ID_SYN_SUP) ]
sending packet: from 192.1.2.45[4500] to 192.1.2.23[4500] (XXX bytes)
received packet: from 192.1.2.23[4500] to 192.1.2.45[4500] (XXX bytes)
parsed IKE_AUTH response 1 [ IDr AUTH SA TSi TSr ]
testing/pluto/interop-ikev2-strongswan-17-delete-sa-responder failed west:output-different
--- MASTER/testing/pluto/interop-ikev2-strongswan-17-delete-sa-responder/west.console.txt
+++ OUTPUT/testing/pluto/interop-ikev2-strongswan-17-delete-sa-responder/west.console.txt
@@ -39,10 +39,9 @@
sending packet: from 192.1.2.45[500] to 192.1.2.23[500] (XXX bytes)
received packet: from 192.1.2.23[500] to 192.1.2.45[500] (XXX bytes)
parsed IKE_SA_INIT response 0 [ SA KE No N(FRAG_SUP) N(NATD_S_IP) N(NATD_D_IP) ]
-sending cert request for "C=CA, ST=Ontario, L=Toronto, O=Libreswan, OU=Test Department, CN=Libreswan test CA for mainca, E=testing at libreswan.org"
authentication of 'west' (myself) with pre-shared key
establishing CHILD_SA westnet-eastnet-ikev2{1}
-generating IKE_AUTH request 1 [ IDi N(INIT_CONTACT) CERTREQ IDr AUTH SA TSi TSr N(MOBIKE_SUP) N(ADD_4_ADDR) N(ADD_4_ADDR) N(EAP_ONLY) N(MSG_ID_SYN_SUP) ]
+generating IKE_AUTH request 1 [ IDi N(INIT_CONTACT) IDr AUTH SA TSi TSr N(MOBIKE_SUP) N(ADD_4_ADDR) N(ADD_4_ADDR) N(EAP_ONLY) N(MSG_ID_SYN_SUP) ]
sending packet: from 192.1.2.45[4500] to 192.1.2.23[4500] (XXX bytes)
received packet: from 192.1.2.23[4500] to 192.1.2.45[4500] (XXX bytes)
parsed IKE_AUTH response 1 [ IDr AUTH SA TSi TSr ]
testing/pluto/interop-ikev2-strongswan-19-x509-res-certreq failed west:output-different
just RETRANSMISSION
testing/pluto/interop-ikev2-strongswan-35-rekey-reauth failed east:output-different west:output-different
east: reqid difference
west:
ipsec status | grep westnet-eastnet-ikev2
-000 "westnet-eastnet-ikev2": 192.0.1.0/24===192.1.2.45<192.1.2.45>[@west]...192.1.2.23<192.1.2.23>[@east]===192.0.2.0/24; erouted; eroute owner: #5
+000 "westnet-eastnet-ikev2": 192.0.1.0/24===192.1.2.45<192.1.2.45>[@west]...192.1.2.23<192.1.2.23>[@east]===192.0.2.0/24; prospective erouted; eroute owner: #0
-000 "westnet-eastnet-ikev2": newest ISAKMP SA: #4; newest IPsec SA: #5;
+000 "westnet-eastnet-ikev2": newest ISAKMP SA: #0; newest IPsec SA: #0;
-000 "westnet-eastnet-ikev2": IKEv2 algorithm newest: AES_CBC_256-HMAC_SHA2_256-MODP2048
000 "westnet-eastnet-ikev2": ESP algorithms: AES_CBC_128-HMAC_SHA2_512_256-MODP2048
-000 "westnet-eastnet-ikev2": ESP algorithm newest: AES_CBC_128-HMAC_SHA2_512_256; pfsgroup=MODP2048
-000 #5: "westnet-eastnet-ikev2":500 STATE_V2_IPSEC_I (IPsec SA established); EVENT_SA_REPLACE in XXs; newest IPSEC; eroute owner; isakmp#4; idle; import:admin initiate
-000 #5: "westnet-eastnet-ikev2" esp.ESPSPIi at 192.1.2.23 esp.ESPSPIi at 192.1.2.45 tun.0 at 192.1.2.23 tun.0 at 192.1.2.45 ref=0 refhim=0 Traffic: ESPin=0B ESPout=0B! ESPmax=0B
-000 #4: "westnet-eastnet-ikev2":500 STATE_PARENT_I3 (PARENT SA established); EVENT_SA_REPLACE in XXs; newest ISAKMP; isakmp#0; idle; import:admin initiate
-000 #4: "westnet-eastnet-ikev2" ref=0 refhim=0 Traffic:
and more
testing/pluto/interop-ikev2-strongswan-35-responder-rekey-pfs passed
New Failure, west:
if [ -f /var/run/charon.pid ]; then strongswan status ; fi
Security Associations (1 up, 0 connecting):
westnet-eastnet-ikev2[1]: ESTABLISHED XXX second ago, 192.1.2.45[west]...192.1.2.23[east]
+westnet-eastnet-ikev2{1}: DELETING, TUNNEL, reqid 1
+westnet-eastnet-ikev2{1}: 192.0.1.0/24 === 192.0.2.0/24
westnet-eastnet-ikev2{2}: DELETING, TUNNEL, reqid 1
westnet-eastnet-ikev2{2}: 192.0.1.0/24 === 192.0.2.0/24
westnet-eastnet-ikev2{3}: INSTALLED, TUNNEL, reqid 1, ESP SPIs: SPISPI_i SPISPI_o
testing/pluto/dnssec-pluto-01 failed west:output-different
-000 "westnet-eastnet-etc-hosts-auto-add": 192.0.1.0/24===192.1.2.45<192.1.2.45>[@west]...192.1.2.23<east-from-hosts-file>[@east]===192.0.2.0/24; unrouted; eroute owner: #0
-000 "westnet-eastnet-etc-hosts-auto-add": oriented; my_ip=unset; their_ip=unset; my_updown=ipsec _updown;
testing/pluto/ikev2-55-ipseckey-02 failed road:output-different
-003 "road-east-2" #1: Can't find the private key from the NSS CKA_ID
-003 "road-east-2" #1: Failed to find our RSA key
-000 "road-east-2" #1: realse whack for IKE SA, but releasing whack for pending IPSEC SA
+003 "road-east-2" #1: Can't find the certificate or private key from the NSS CKA_ID
+003 "road-east-2" #1: DigSig: failed to find our RSA key
+000 "road-east-2" #1: release whack for IKE SA, but releasing whack for pending IPSEC SA
[FIXED]
testing/pluto/l2tp-01 failed north:output-different
different data byte counts
testing/pluto/l2tp-02 failed north:output-different
different data byte counts
testing/pluto/nss-cert-04 failed west:output-different
discarding duplicate packet
testing/pluto/nss-cert-05 failed west:output-different
discarding duplicate packet
testing/pluto/nss-cert-nosecret failed west:output-different
lots of this:
-000 TIMESTAMP, 1024 RSA Key AwXXXXXXX (has private key), until TIMESTAMP ok
+000 TIMESTAMP, 1024 RSA Key AwXXXXXXX (no private key), until TIMESTAMP ok
testing/pluto/nss-cert-09-notyetvalid-initiator failed east:output-different west:output-different
east:
-"nss-cert" #1: ERROR: Peer's Certificate has expired.
west:
-/testing/guestbin/swan-prep --x509 --x509name notyetvalid
+/testing/guestbin/swan-prep --x509
[Paul 8a0b5b79]
testing/pluto/nss-cert-10-notyetvalid-responder-ikev2 failed east:output-different west:output-different
east:
grep "ERROR" /tmp/pluto.log
+"nss-cert" #2: ERROR: netlink response for Del SA esp.ESPSPIi at 192.1.2.45 included errno 3: No such process
+"nss-cert" #2: ERROR: netlink response for Del SA esp.ESPSPIi at 192.1.2.23 included errno 3: No such process
west:
002 "nss-cert" #2: suppressing retransmit because IMPAIR_RETRANSMITS is set.
-003 "nss-cert" #2: Certificate E=testing at libreswan.org,CN=notyetvalid.testing.libreswan.org,OU=Test Department,O=Libreswan,L=Toronto,ST=Ontario,C=CA failed verification
-003 "nss-cert" #2: ERROR: Peer's Certificate has expired.
-002 "nss-cert" #2: IKEv2 mode peer ID is ID_DER_ASN1_DN: 'C=CA, ST=Ontario, L=Toronto, O=Libreswan, OU=Test Department, CN=notyetvalid.testing.libreswan.org, E=testing at libreswan.org'
-003 "nss-cert" #2: no RSA public key known for 'C=CA, ST=Ontario, L=Toronto, O=Libreswan, OU=Test Department, CN=notyetvalid.testing.libreswan.org, E=testing at libreswan.org'
-002 "nss-cert" #2: RSA authentication failed
-224 "nss-cert" #2: STATE_PARENT_I2: v2N_AUTHENTICATION_FAILED
-003 "nss-cert" #2: EXPECTATION FAILED: st != NULL && st->st_event != NULL && st->st_event->ev_type == EVENT_v2_RETRANSMIT (in complete_v2_state_transition at /source/programs/pluto/ikev2.c:1827)
grep "ERROR" /tmp/pluto.log
-"nss-cert" #2: ERROR: Peer's Certificate has expired.
testing/pluto/ipsec-hostkey-ckaid-01 passed
testing/pluto/ipsec-hostkey-ckaid-02-fips failed west:output-different
ipsec newhostkey
-Generated RSA key pair with CKAID <<CKAID#1>> was stored in the NSS database
+FIPS HMAC integrity verification test failed.
ipsec showhostkey --list
-< 1> RSA keyid: <<KEYID#1>> ckaid: <<CKAID#1>>
ipsec showhostkey --left --ckaid $ckaid
- # rsakey <<KEYID#1>>
- leftrsasigkey=<<RSASIGKEY#1>>
+PATH/libexec/ipsec/showhostkey: option '--ckaid' requires an argument
NEW:
One minor problem is that an empty $ckid looks like a missing parameter.
I changed the reference to have quotes. So the error message should be
something else (the argument is there, but empty). But the message
didn't change. What's up????
I manually tested this on a different machine and got a different error
ipsec showhostkey --left --ckaid "$ckaid"
- # rsakey <<KEYID#1>>
- leftrsasigkey=<<RSASIGKEY#1>>
+PATH/libexec/ipsec/showhostkey: option '--ckaid' requires an argument
More information about the Swan-dev
mailing list