[Swan-dev] a scan of failing tests

[Antony Antony]

may be wrong files got pushed into the master! ikev2-liveness-01 and ikev2-liveness-05 are failing on e6a4bc7

ikev2-liveness-01  wrong console files {west,east}-console.txt
ikev2-liveness-05 diff lines.

-antony

On Sat, Jul 09, 2016 at 12:22:39AM +0200, Paul Wouters wrote:
> Ah yes, three new files weren't added to git yet. Pushed now
> 
> Sent from my iPhone
> 
> > On Jul 8, 2016, at 23:53, Antony Antony <antony at phenome.org> wrote:
> > 
> > good to see tests are cleaned up. Thanks! From the last run, some of the ikev2-liveness-0x tests still show up as failed. May be someone forgot to push the right branch.
> > 
> > for example
> > ikev2-liveness-01 west has a big diff
> > ikev2-liveness-03 missing baseline
> > ikev2-liveness-05 west has a diff
> > 
> > --- west.console.txt    2016-07-07 20:35:21.436057064 +0200
> > +++ OUTPUT/west.console.txt    2016-07-08 11:59:20.596439126 +0200
> > @@ -80,6 +80,8 @@
> > west #
> >  ping -w 2 -q -n -c 3 -I 192.0.1.254 192.0.2.254
> > PING 192.0.2.254 (192.0.2.254) from 192.0.1.254 : 56(84) bytes of data.
> > +[ 00.00] IN= OUT=eth1 SRC=192.0.1.254 DST=192.0.2.254 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=34696 DF PROTO=ICMP TYPE=8 CODE=0 ID=2458 SEQ=1 
> > +[ 00.00] IN= OUT=eth1 SRC=192.0.1.254 DST=192.0.2.254 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=34856 DF PROTO=ICMP TYPE=8 CODE=0 ID=2458 SEQ=1 
> > --- 192.0.2.254 ping statistics ---
> > 0 packets transmitted, 0 received
> > west #
> > 
> >> On Fri, Jul 08, 2016 at 05:32:24PM -0400, Paul Wouters wrote:
> >> On Wed, 6 Jul 2016, D. Hugh Redelmeier wrote:
> >> 
> >> I've gone through these and just commited the fixes listed below.
> >> 
> >> Some issues do need to get looked at before 3.18
> >> 
> >> Paul
> >> 
> >> 
> >> basic-pluto-00 failed west:output-different <== no hope of working
> >> 
> >> FIXED ikev2-delete-02 failed east:output-different west:output-different <== reference logs obsolete
> >> LEAK REPORT ADDED ikev2-12-x509-ikev1-rw failed west:output-different <=== leak reported
> >> EAST DID NOT PROCESS TRANSPORT NOTIFY ikev2-12-transport-psk failed east:output-different west:output-different <== local policy requires Transport Mode but peer requires required Tunnel Mode
> >> FIXED ikev2-32-nat-rw-rekey failed east:EXPECTATION road:output-different <== lease newly reported???
> >> GROUTED POLICY NEW? newoe-25-cat-2 failed road:output-different <== west conns not loaded?
> >> FIXED newoe-25-cat-3-4-way failed east:output-different north:output-different road:output-different west:output-different <=== road diagram changed
> >> EPHEMERAL newoe-25-cat-4 failed east:output-different road:output-different west:output-different <== not clear
> >> CLEAR policy for nic and host added ?? newoe-25-cat-5 failed east:output-different road:output-different <=== not clear
> >> EPHEMERAL dpd-01 failed west:output-different <== timing of SA?
> >> EPHEMERAL plus ping fix dpd-04 failed east:output-different west:output-different <== address and port changes?
> >> EPHEMERAL plus ping fix dpd-05 failed west:output-different <== port change
> >> EPHEMERAL plus ping fix dpd-06 failed west:output-different <== new bad iptables rules?
> >> FIXED SINCE ikev2-liveness-05 failed east:output-different west:output-different <== no longer KLIPS?
> >> FIXED SINCE ikev2-liveness-06 failed east:EXPECTATION,output-different road:output-different <== no longer KLIPS?
> >> FIXED SINCE ikev2-liveness-07 failed east:EXPECTATION,output-different road:output-different <== no longer KLIPS?
> >> FIXED SINCE ikev2-liveness-08-drop failed east:output-different west:output-different <== no longer KLIPS?
> >> KNOWN refhim= delete bug delete-sa-03 failed east:output-different west:output-different <== refhim
> >> FIXED x509-pluto-02 failed north:output-different <== no outBytes
> >> WORKSFORME  x509-pluto-frag-01 failed road:output-different <= retransmission message
> >> WORKSFORME nat-pluto-04 failed east:output-different road:output-different <== ?
> >> FIXED ikev1-algo-ike-aes-02 failed east:output-different west:output-different <== no longer KLIPS
> >> WORKSFORME psk-pluto-02 incomplete east:output-different road:output-different <== INVALID_HASH_INFORMATION ?
> >> FIXED xauth-pluto-12 failed east:output-different road:output-different <== no longer KLIPS
> >> FIXED xauth-pluto-15 failed east:output-different road:output-different <== no longer KLIPS
> >> EPHEMERAL xauth-pluto-17 failed east:output-different <== swapped IP in XFRM ?
> >> EPHEMERAL xauth-pluto-18 failed east:output-different <== swapped IP in XFRM ?
> >> NEEDS INVESTIGATINH basic-pluto-12 failed east:output-different west:output-different <== lots of additional activity
> >> FIXED SINCE interop-ikev2-strongswan-16-ah-initiator-sha512 failed west:output-different <== cert req changed?
> >> WORKSFORME interop-ikev2-strongswan-17-ah-initiator-sha256 failed west:output-different <== cert req changed?
> >> FIXED SINCE and WORKSFORME interop-ikev1-strongswan-15-twofish failed east:output-different west:output-different <== message ignored because it contains an unexpected payload type (ISAKMP_NEXT_HASH)
> >> FIXED SINCE and WORKSFORME interop-ikev1-strongswan-16-serpent failed east:output-different west:output-different <== message ignored because it contains an unexpected payload type (ISAKMP_NEXT_HASH)
> >> WORKSFORME ipv6-tunnel-mode-02-netkey-netkey failed east:output-different west:output-different <== failure to communicate
> >> WORKSFORME ipv6-tunnel-mode-04-rw failed east:output-different road:output-different <== failure to communicate
> >> NEEDS INVESTIGATION interop-ikev2-racoon-02-psk-responder failed east:output-different west:output-different <== ?
> >> WORKSFORME interop-ikev2-strongswan-03-psk-initiator failed west:output-different <== cert request different [but PSK??]
> >> WORKSFORME  interop-ikev2-strongswan-04-x509-responder failed west:output-different <== failed to negoriate
> >> WORKSFORME interop-ikev2-strongswan-07-strongswan failed west:output-different <== cert request different
> >> WORKSFORME interop-ikev2-strongswan-13-ah-initiator failed west:output-different <== cert request different
> >> WORKSFORME interop-ikev2-strongswan-14-delete-sa failed west:output-different <== cert request different
> >> NEEDS INVESTIGATION interop-ikev2-strongswan-15-create_child_sa incomplete east:output-different west:output-different <== ss cannot parse ls message
> >> WORKSFORME interop-ikev2-strongswan-17-delete-sa-responder failed west:output-different <== cert request different
> >> WORKSFORME  interop-ikev2-strongswan-27-fragmentation failed west:output-different <== negotiation failed
> >> FIXED SINCE interop-ikev2-strongswan-28-reauth failed west:output-different <== cert request different
> >> NEEDS INVESTIGATION (auto=add conn failed!) dnssec-pluto-01 failed west:output-different <== westnet-eastnet-etc-hosts-auto-add missing
> >> NEEDS INVESTIGATION (but old bug) l2tp-01 failed east:output-different north:output-different <== test changed?  No longer KLIPS?
> >> NEEDS INVESTIGATION (but old bug) l2tp-02 failed east:output-different north:output-different <== test changed?  No longer KLIPS?
> >> DIDNT START fips-08-ikev2-x509 failed east:output-different west:output-different <== test changed?  No longer KLIPS?
> >> WORKSFORME nss-cert-04 failed west:output-different <== ignoring retry?
> >> RERUN nss-cert-05 failed east:EXPECTATION west:output-different <== ignoring retry?
> >> RERUN nss-cert-08-mismatch failed west:EXPECTATION,output-different <== logging seems to have changed
> >> RERUN nss-cert-chain-02 failed west:output-different <== retransmission different
> >> RERUN nss-cert-crl-02 failed west:output-different <== duplicate packet
> >> RERUN nss-cert-nosecret failed east:output-different west:output-different <== certs or keys different?
> >> RERUN nss-cert-ocsp-01-strict failed west:output-different <== negotiation failed
> >> RERUN nss-cert-ocsp-02 failed east:output-different west:output-different <== no "certificate revoked"; cert accepted!!
> >> RERUN nss-cert-ocsp-02-ikev2 failed east:output-different west:output-different <== no "certificate revoked"; cert accepted!!
> >> RERUN nss-cert-ocsp-02-strict failed east:output-different west:output-different <== no "certificate revoked"; cert rejected for a different reason
> >> RERUN nss-cert-ocsp-03-strict failed west:output-different <== lots more retransmissions
> >> RERUN nss-cert-ocsp-04 failed west:output-different <== negotiation succeeded (mistake??)
> >> RERUN nss-cert-ocsp-04-strict failed west:output-different <= old: INVALID_ID_INFORMATION, new: INVALID_KEY_INFORMATION ???
> >> RERUN nss-cert-ocsp-05-strict failed west:output-different <== negotiation failure when success is expected
> >> RERUN nss-cert-ocsp-06 failed west:output-different <== negotiation success when failure expected
> >> RERUN nss-cert-ocsp-07-nourl failed east:output-different west:output-different <== no "certificate revoked!", negotiation unexpectedly succeeded
> >> 
> >> 
> >> 
> >> _______________________________________________
> >> Swan-dev mailing list
> >> Swan-dev at lists.libreswan.org
> >> https://lists.libreswan.org/mailman/listinfo/swan-dev
>