[Swan-dev] last night's test failures
D. Hugh Redelmeier
hugh at mimosa.com
Sun Sep 6 16:27:17 EEST 2015
| From: D. Hugh Redelmeier <hugh at mimosa.com>
| Subject: [Swan-dev] ikev2-12-x509-ikev1* tests fail
|
| I imagine that somebody changed something without updating the
| reference logs.
Still true about ikev2-12-x509-ikev1 and ikev2-12-x509-ikev-rw
Since last night's run completed, I have more observations.
Unless noted, I only report changes from the previous complete run.
================
ikev2-48-nat-cp is new, and it fails:
-006 #2: "westnet-eastnet-ipv4-psk-ikev2"[1] 192.1.2.23, type=ESP, add_time=1234567890, outBytes=168, inBytes=168, id='@east'
+006 #2: "westnet-eastnet-ipv4-psk-ikev2"[1] 192.1.2.23, type=ESP, add_time=1234567890, inBytes=168, outBytes=168, id='@east'
Guess: reference logs need updating to reflect traffic-reporting fixes
================
nflog tests seem unstable.
nflog-01-global: passed (failed the previous time)
nflog-02-conn: failed with slightly different traffic (9 packets instead of 8)
nflog-03-conns: failed (failed previous time too) with packet capture
and tcpdump completion at a different point in the log
================
newoe-18-cop-block: (passed previously) 3 packets instead of two
newoe-18-private-clear: +169.254.0.0/16 dev eth0 scope link metric 1002
================
netkey-algo-camellia-01: +169.254.0.0/16 dev eth0 scope link metric 1002
================================================================
Here's the complete list of tests failures. Since most are not new I
have not investigated them. But we should get them fixed.
- Known problem: my strongswan is out of date (where do I get an update?).
- I include "wip" and "skiptest" entries.
kvmplutotest netkey-audit-01 good!=bad,west:bad,east:bad
kvmplutotest ikev1-initial-contact wip!=bad,west:missing-baseline,east:missing-baseline
kvmplutotest ikev2-01-fallback-ikev1 good!=bad,west:bad,east:bad
kvmplutotest ikev2-delete-01 good!=bad,west:bad,east:ok
kvmplutotest ikev2-delete-02 good!=bad,west:bad,east:ok
kvmplutotest ikev2-delete-03-valgrind bad,west:missing-baseline,east:missing-baseline
kvmplutotest ikev2-delete-04 wip!=bad,west:missing-baseline,east:missing-baseline
kvmplutotest ikev2-12-x509-ikev1 good!=bad,west:bad,east:ok
kvmplutotest ikev2-12-x509-ikev1-rw good!=bad,west:bad,east:ok
kvmplutotest ikev2-12-transport-psk good!=bad,west:bad,east:bad
kvmplutotest ikev2-18-x509-alias good!=bad,east:ok,road:bad
kvmplutotest ikev2-19-x509-auto-start good!=bad,east:missing-baseline,road:missing-baseline
kvmplutotest ikev2-41-rw-replace good!=bad,east:ok,road:bad
kvmplutotest ikev2-42-rw-replace-responder good!=bad,east:ok,road:bad
kvmplutotest ikev2-45-impair-gx-02 good!=bad,EXPECT:west.pluto.log,west:bad,east:ok
kvmplutotest ikev2-46-basic-psk-netkey good!=bad,west:bad,east:bad
kvmplutotest ikev2-47-priority wip!=bad,west:missing-baseline,east:missing-baseline
kvmplutotest ikev2-48-nat-cp good!=bad,east:ok,road:bad
kvmplutotest ikev2-24-cryptoload wip!=bad,west:missing-baseline,east:missing-baseline
kvmplutotest ikev2-27-uniqueid wip!=bad,east:missing-baseline,north:missing-baseline
kvmplutotest ikev2-29-no-rekey good!=bad,west:bad,east:ok
kvmplutotest ikev2-ddns-02 good!=bad,west:bad,east:bad
kvmplutotest ikev2-04-basic-x509-nhelpers0 good!=bad,west:bad,east:bad
kvmplutotest ikev2-10-2behind-nat wip!=bad,east:missing-baseline,road:missing-baseline
kvmplutotest ikev2-algo-sha2-05 good!=bad,west:bad,east:bad
kvmplutotest ikev2-algo-sha2-07 wip!=bad,west:missing-baseline,east:missing-baseline
kvmplutotest ikev1-impair-gx-02 good!=bad,west:missing-baseline,east:missing-baseline
kvmplutotest ikev2-minor-version-initiator good!=bad,west:ok,east:bad
kvmplutotest ikev2-isakmp-reserved-flags-01 good!=bad,west:bad,east:bad
kvmplutotest ikev2-payload-reserved-flags-01 good!=bad,west:bad,east:bad
kvmplutotest ikev2-allow-narrow-01 good!=bad,EXPECT:west.pluto.log,west:bad,east:ok
kvmplutotest ikev2-allow-narrow-02 good!=bad,EXPECT:west.pluto.log,west:bad,east:ok
kvmplutotest ikev2-allow-narrow-03 good!=bad,EXPECT:west.pluto.log,west:bad,east:ok
kvmplutotest ikev2-ddos-01 wip!=bad,east:missing-baseline,road:missing-baseline
kvmplutotest ikev2-switchnat-01 wip!=bad,east:missing-baseline,road:missing-baseline
kvmplutotest ikev1-switchnat-01 good!=bad,east:missing-baseline,road:missing-baseline
kvmplutotest ikev1-connswitch-ports-01 wip!=bad,west:missing-baseline,east:missing-baseline
kvmplutotest nflog-02-conn good!=bad,west:bad,east:ok
kvmplutotest nflog-03-conns good!=bad,west:bad,east:ok
kvmplutotest ikev2-43-init-retransmit good!=bad,west:bad,east:bad
kvmplutotest ikev2-frag-02-ipv6 good!=bad,west:bad,east:bad
kvmplutotest newoe-01-whack good!=bad,west:bad,east:ok
kvmplutotest newoe-02-klips wip!=bad,EXPECT:road.pluto.log,east:missing-baseline,road:missing-baseline
kvmplutotest newoe-04-01 good!=none,NO-OUTPUT
kvmplutotest newoe-05-hold-pass good!=bad,east:ok,road:bad
kvmplutotest newoe-06-prio good,EXPECT:road.pluto.log
kvmplutotest newoe-08-ike-replace-responder good!=bad,east:ok,road:bad
kvmplutotest newoe-15-portpass good!=bad,east:ok,road:bad
kvmplutotest newoe-16-pass-hold good!=bad,east:ok,road:bad
kvmplutotest newoe-17-block-in-clear-clear good!=bad,east:missing-baseline,road:missing-baseline
kvmplutotest newoe-18-poc-block good!=bad,east:ok,road:bad
kvmplutotest newoe-18-cop-block good!=bad,east:ok,road:bad
kvmplutotest newoe-18-private-clear good!=bad,east:bad,road:ok
kvmplutotest newoe-18-private-block good!=bad,east:ok,road:bad
kvmplutotest newoe-19-poc-poc-clear wip!=bad,east:missing-baseline,road:missing-baseline
ctltest pluto-unit-01 good!=none,NO-OUTPUT
ctltest pluto-unit-02 good!=none,NO-OUTPUT
ctltest pluto-dontreky-expiry-01 good!=none,NO-OUTPUT
kvmplutotest dpd-01-netkey good!=bad,EXPECT:west.pluto.log,west:bad,east:ok
kvmplutotest dpd-04 good!=bad,west:bad,east:ok
kvmplutotest dpd-08 wip!=bad,west:missing-baseline,east:missing-baseline
kvmplutotest dpd-08-netkey wip!=bad,west:missing-baseline,east:missing-baseline
kvmplutotest ikev2-liveness-06 good,EXPECT:east.pluto.log
kvmplutotest ikev2-liveness-07 good!=bad,EXPECT:east.pluto.log,east:bad,road:bad
kvmplutotest delete-sa-03 wip!=bad,west:bad,east:bad
kvmplutotest delete-state-01 good!=bad,west:bad,east:ok
kvmplutotest ikev1-rekey-connswitch wip!=bad,west:bad,east:bad
ctltest whack-status-01 good!=none,NO-OUTPUT
ctltest vpn-dns-01 good!=none,NO-OUTPUT
kvmplutotest x509-pluto-01 good!=bad,west:bad,east:bad
kvmplutotest x509-pluto-02 good!=bad,east:ok,north:bad
kvmplutotest x509-pluto-03 good!=bad,west:bad,east:bad
kvmplutotest nat-pluto-10 good!=bad,EXPECT:east.pluto.log,east:missing-baseline,north:missing-baseline
kvmplutotest nat-dpd-pluto-01 wip!=bad,east:missing-baseline,north:missing-baseline
skiptest xauth-pluto-17 wip!=none,NO-OUTPUT
kvmplutotest xauth-pluto-20 good!=none,NO-OUTPUT
kvmplutotest aggr-pluto-03 wip!=none,NO-OUTPUT
kvmplutotest basic-pluto-10 wip!=bad,west:missing-baseline,east:missing-baseline
kvmplutotest basic-pluto-01-valgrind good!=bad,west:missing-baseline,east:missing-baseline
kvmplutotest basic-pluto-11 good!=bad,west:bad,east:bad
kvmplutotest basic-pluto-12-netkey wip!=bad,west:missing-baseline,east:missing-baseline
kvmplutotest basic-pluto-14-klips-route wip!=bad,west:missing-baseline,east:missing-baseline
kvmplutotest x509-chain-01 good!=bad,west:bad,east:bad
kvmplutotest x509-chain-02 good!=bad,west:bad,east:bad
kvmplutotest x509-chain-03 good!=bad,west:bad,east:bad
kvmplutotest netkey-algo-camellia-01 good!=bad,west:bad,east:ok
kvmplutotest algo-pluto-04 wip!=bad,west:missing-baseline,east:missing-baseline
kvmplutotest algo-pluto-13-invalid-3des good!=bad,west:bad,east:ok
kvmplutotest interop-ikev2-strongswan-16-ah-initiator-sha512 good!=bad,west:bad,east:ok
kvmplutotest interop-ikev2-strongswan-17-ah-initiator-sha256 good!=bad,west:bad,east:ok
kvmplutotest interop-ikev2-strongswan-22-cp-responder-psk good!=bad,east:ok,road:bad
kvmplutotest interop-ikev2-strongswan-26-ke-mismatch-responder good!=bad,west:bad,east:ok
kvmplutotest netkey-pluto-07 wip!=bad,west:bad,east:bad
kvmplutotest labeled-ipsec-01 wip!=bad,west:missing-baseline,east:missing-baseline
kvmplutotest ipv6-v6-through-v6-klips-klips incomplete!=bad,west:bad,east:bad
kvmplutotest ipv6-transport-mode-01-klips-klips incomplete!=bad,west:bad,east:bad
kvmplutotest ipv6-transport-mode-02-netkey-netkey incomplete!=bad,west:bad,east:bad
kvmplutotest ipv6-transport-mode-03-klips-netkey incomplete!=bad,west:bad,east:bad
kvmplutotest ipv6-tunnel-mode-01-klips-klips incomplete!=bad,west:bad,east:bad
kvmplutotest ipv6-tunnel-mode-02-netkey-netkey good!=bad,west:bad,east:bad
kvmplutotest ipv6-tunnel-mode-03-klips-netkey incomplete!=bad,west:bad,east:bad
kvmplutotest ipv6-transport-ts-mode-04-netkey-netkey incomplete!=bad,west:bad,east:bad
kvmplutotest ipv6-tunnel-mode-03-rw wip!=bad,east:missing-baseline,road:missing-baseline
kvmplutotest ipv6-tunnel-mode-04-rw good!=bad,east:bad,road:bad
kvmplutotest ikev2-ipv6-transport-mode-02-netkey-netkey wip!=bad,east:missing-baseline,road:missing-baseline
kvmplutotest interop-ikev2-racoon-01-noconn good!=bad,west:missing-baseline,east:missing-baseline
kvmplutotest interop-ikev2-racoon-03-psk-initiator good!=none,NO-OUTPUT
kvmplutotest interop-ikev2-racoon-04-x509-responder wip!=bad,west:missing-baseline,east:missing-baseline
kvmplutotest interop-ikev2-racoon-05-x509-initiator good!=none,NO-OUTPUT
kvmplutotest interop-ikev2-strongswan-03-psk-initiator good!=bad,west:bad,east:ok
kvmplutotest interop-ikev2-strongswan-04-x509-responder good!=bad,west:bad,east:ok
kvmplutotest interop-ikev2-strongswan-07-strongswan good!=bad,west:bad,east:ok
kvmplutotest interop-ikev2-strongswan-10-nat-initiator good!=bad,east:ok,road:bad
kvmplutotest interop-ikev2-strongswan-11-nat-initiator good!=bad,east:ok,road:bad
kvmplutotest interop-ikev2-strongswan-13-ah-initiator good!=bad,west:bad,east:ok
kvmplutotest interop-ikev2-strongswan-14-delete-sa good!=bad,west:bad,east:ok
kvmplutotest interop-ikev2-strongswan-15-create_child_sa good!=bad,west:bad,east:bad
kvmplutotest interop-ikev2-strongswan-17-delete-sa-responder good!=bad,west:bad,east:ok
kvmplutotest interop-ikev2-strongswan-19-x509-res-certreq good!=bad,west:bad,east:bad
kvmplutotest interop-ikev2-strongswan-27-fragmentation good!=bad,west:bad,east:ok
kvmplutotest x509-pluto-frag-00 good!=bad,east:bad,road:ok
kvmplutotest x509-pluto-frag-01 good!=bad,east:bad,road:ok
kvmplutotest x509-pluto-frag-03 good!=bad,east:bad,road:ok
kvmplutotest x509-pluto-frag-04 good!=bad,east:bad,road:ok
kvmplutotest ikeport-01 wip!=bad,west:missing-baseline,east:missing-baseline
skiptest fips-05-ikev1-gcm wip!=none,NO-OUTPUT
More information about the Swan-dev
mailing list