[Swan-dev] a scan of failing tests

[Antony Antony]

good to see tests are cleaned up. Thanks! From the last run, some of the ikev2-liveness-0x tests still show up as failed. May be someone forgot to push the right branch.

for example
ikev2-liveness-01 west has a big diff
ikev2-liveness-03 missing baseline
ikev2-liveness-05 west has a diff

--- west.console.txt	2016-07-07 20:35:21.436057064 +0200
+++ OUTPUT/west.console.txt	2016-07-08 11:59:20.596439126 +0200
@@ -80,6 +80,8 @@
 west #
  ping -w 2 -q -n -c 3 -I 192.0.1.254 192.0.2.254
 PING 192.0.2.254 (192.0.2.254) from 192.0.1.254 : 56(84) bytes of data.
+[ 00.00] IN= OUT=eth1 SRC=192.0.1.254 DST=192.0.2.254 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=34696 DF PROTO=ICMP TYPE=8 CODE=0 ID=2458 SEQ=1 
+[ 00.00] IN= OUT=eth1 SRC=192.0.1.254 DST=192.0.2.254 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=34856 DF PROTO=ICMP TYPE=8 CODE=0 ID=2458 SEQ=1 
 --- 192.0.2.254 ping statistics ---
 0 packets transmitted, 0 received
 west #

On Fri, Jul 08, 2016 at 05:32:24PM -0400, Paul Wouters wrote:
> On Wed, 6 Jul 2016, D. Hugh Redelmeier wrote:
> 
> I've gone through these and just commited the fixes listed below.
> 
> Some issues do need to get looked at before 3.18
> 
> Paul
> 
> 
> basic-pluto-00 failed west:output-different <== no hope of working
> 
> FIXED ikev2-delete-02 failed east:output-different west:output-different <== reference logs obsolete
> LEAK REPORT ADDED ikev2-12-x509-ikev1-rw failed west:output-different <=== leak reported
> EAST DID NOT PROCESS TRANSPORT NOTIFY ikev2-12-transport-psk failed east:output-different west:output-different <== local policy requires Transport Mode but peer requires required Tunnel Mode
> FIXED ikev2-32-nat-rw-rekey failed east:EXPECTATION road:output-different <== lease newly reported???
> GROUTED POLICY NEW? newoe-25-cat-2 failed road:output-different <== west conns not loaded?
> FIXED newoe-25-cat-3-4-way failed east:output-different north:output-different road:output-different west:output-different <=== road diagram changed
> EPHEMERAL newoe-25-cat-4 failed east:output-different road:output-different west:output-different <== not clear
> CLEAR policy for nic and host added ?? newoe-25-cat-5 failed east:output-different road:output-different <=== not clear
> EPHEMERAL dpd-01 failed west:output-different <== timing of SA?
> EPHEMERAL plus ping fix dpd-04 failed east:output-different west:output-different <== address and port changes?
> EPHEMERAL plus ping fix dpd-05 failed west:output-different <== port change
> EPHEMERAL plus ping fix dpd-06 failed west:output-different <== new bad iptables rules?
> FIXED SINCE ikev2-liveness-05 failed east:output-different west:output-different <== no longer KLIPS?
> FIXED SINCE ikev2-liveness-06 failed east:EXPECTATION,output-different road:output-different <== no longer KLIPS?
> FIXED SINCE ikev2-liveness-07 failed east:EXPECTATION,output-different road:output-different <== no longer KLIPS?
> FIXED SINCE ikev2-liveness-08-drop failed east:output-different west:output-different <== no longer KLIPS?
> KNOWN refhim= delete bug delete-sa-03 failed east:output-different west:output-different <== refhim
> FIXED x509-pluto-02 failed north:output-different <== no outBytes
> WORKSFORME  x509-pluto-frag-01 failed road:output-different <= retransmission message
> WORKSFORME nat-pluto-04 failed east:output-different road:output-different <== ?
> FIXED ikev1-algo-ike-aes-02 failed east:output-different west:output-different <== no longer KLIPS
> WORKSFORME psk-pluto-02 incomplete east:output-different road:output-different <== INVALID_HASH_INFORMATION ?
> FIXED xauth-pluto-12 failed east:output-different road:output-different <== no longer KLIPS
> FIXED xauth-pluto-15 failed east:output-different road:output-different <== no longer KLIPS
> EPHEMERAL xauth-pluto-17 failed east:output-different <== swapped IP in XFRM ?
> EPHEMERAL xauth-pluto-18 failed east:output-different <== swapped IP in XFRM ?
> NEEDS INVESTIGATINH basic-pluto-12 failed east:output-different west:output-different <== lots of additional activity
> FIXED SINCE interop-ikev2-strongswan-16-ah-initiator-sha512 failed west:output-different <== cert req changed?
> WORKSFORME interop-ikev2-strongswan-17-ah-initiator-sha256 failed west:output-different <== cert req changed?
> FIXED SINCE and WORKSFORME interop-ikev1-strongswan-15-twofish failed east:output-different west:output-different <== message ignored because it contains an unexpected payload type (ISAKMP_NEXT_HASH)
> FIXED SINCE and WORKSFORME interop-ikev1-strongswan-16-serpent failed east:output-different west:output-different <== message ignored because it contains an unexpected payload type (ISAKMP_NEXT_HASH)
> WORKSFORME ipv6-tunnel-mode-02-netkey-netkey failed east:output-different west:output-different <== failure to communicate
> WORKSFORME ipv6-tunnel-mode-04-rw failed east:output-different road:output-different <== failure to communicate
> NEEDS INVESTIGATION interop-ikev2-racoon-02-psk-responder failed east:output-different west:output-different <== ?
> WORKSFORME interop-ikev2-strongswan-03-psk-initiator failed west:output-different <== cert request different [but PSK??]
> WORKSFORME  interop-ikev2-strongswan-04-x509-responder failed west:output-different <== failed to negoriate
> WORKSFORME interop-ikev2-strongswan-07-strongswan failed west:output-different <== cert request different
> WORKSFORME interop-ikev2-strongswan-13-ah-initiator failed west:output-different <== cert request different
> WORKSFORME interop-ikev2-strongswan-14-delete-sa failed west:output-different <== cert request different
> NEEDS INVESTIGATION interop-ikev2-strongswan-15-create_child_sa incomplete east:output-different west:output-different <== ss cannot parse ls message
> WORKSFORME interop-ikev2-strongswan-17-delete-sa-responder failed west:output-different <== cert request different
> WORKSFORME  interop-ikev2-strongswan-27-fragmentation failed west:output-different <== negotiation failed
> FIXED SINCE interop-ikev2-strongswan-28-reauth failed west:output-different <== cert request different
> NEEDS INVESTIGATION (auto=add conn failed!) dnssec-pluto-01 failed west:output-different <== westnet-eastnet-etc-hosts-auto-add missing
> NEEDS INVESTIGATION (but old bug) l2tp-01 failed east:output-different north:output-different <== test changed?  No longer KLIPS?
> NEEDS INVESTIGATION (but old bug) l2tp-02 failed east:output-different north:output-different <== test changed?  No longer KLIPS?
> DIDNT START fips-08-ikev2-x509 failed east:output-different west:output-different <== test changed?  No longer KLIPS?
> WORKSFORME nss-cert-04 failed west:output-different <== ignoring retry?
> RERUN nss-cert-05 failed east:EXPECTATION west:output-different <== ignoring retry?
> RERUN nss-cert-08-mismatch failed west:EXPECTATION,output-different <== logging seems to have changed
> RERUN nss-cert-chain-02 failed west:output-different <== retransmission different
> RERUN nss-cert-crl-02 failed west:output-different <== duplicate packet
> RERUN nss-cert-nosecret failed east:output-different west:output-different <== certs or keys different?
> RERUN nss-cert-ocsp-01-strict failed west:output-different <== negotiation failed
> RERUN nss-cert-ocsp-02 failed east:output-different west:output-different <== no "certificate revoked"; cert accepted!!
> RERUN nss-cert-ocsp-02-ikev2 failed east:output-different west:output-different <== no "certificate revoked"; cert accepted!!
> RERUN nss-cert-ocsp-02-strict failed east:output-different west:output-different <== no "certificate revoked"; cert rejected for a different reason
> RERUN nss-cert-ocsp-03-strict failed west:output-different <== lots more retransmissions
> RERUN nss-cert-ocsp-04 failed west:output-different <== negotiation succeeded (mistake??)
> RERUN nss-cert-ocsp-04-strict failed west:output-different <= old: INVALID_ID_INFORMATION, new: INVALID_KEY_INFORMATION ???
> RERUN nss-cert-ocsp-05-strict failed west:output-different <== negotiation failure when success is expected
> RERUN nss-cert-ocsp-06 failed west:output-different <== negotiation success when failure expected
> RERUN nss-cert-ocsp-07-nourl failed east:output-different west:output-different <== no "certificate revoked!", negotiation unexpectedly succeeded
> 
> 
> 
> _______________________________________________
> Swan-dev mailing list
> Swan-dev at lists.libreswan.org
> https://lists.libreswan.org/mailman/listinfo/swan-dev