On Sun, 6 Sep 2015, D. Hugh Redelmeier wrote:
> ================
>
> ikev2-48-nat-cp is new, and it fails:
>
> -006 #2: "westnet-eastnet-ipv4-psk-ikev2"[1] 192.1.2.23, type=ESP, add_time=1234567890, outBytes=168, inBytes=168, id='@east'
> +006 #2: "westnet-eastnet-ipv4-psk-ikev2"[1] 192.1.2.23, type=ESP, add_time=1234567890, inBytes=168, outBytes=168, id='@east'
>
> Guess: reference logs need updating to reflect traffic-reporting fixes
Fixed. It didn't yet take into account commit 138e215
> ================
>
> nflog tests seem unstable.
>
> nflog-01-global: passed (failed the previous time)
>
> nflog-02-conn: failed with slightly different traffic (9 packets instead of 8)
the 8 vs 9 is a ping artifact, according to the man page:
-w deadline
Specify a timeout, in seconds, before ping exits regardless of how many packets have been
sent or received. In this case ping does not stop after count packet are sent, it waits
either for deadline expire or until count probes are answered or for some error notification
from network.
So ping -c 4 waits 8 roundtrip times, but ping -c 4 -w 2 will not. And
can cause a plus or minutes 1 packet. I think our best bet is to
sanitize this out :/
> nflog-03-conns: failed (failed previous time too) with packet capture
> and tcpdump completion at a different point in the log
That test output probably needs updating. perhaps we should write a
wrapper to daemonize things and die quietly in the background.
> ================
>
> newoe-18-cop-block: (passed previously) 3 packets instead of two
same ping issue as above
> newoe-18-private-clear: +169.254.0.0/16 dev eth0 scope link metric 1002
> netkey-algo-camellia-01: +169.254.0.0/16 dev eth0 scope link metric 1002
These I see sometimes too. As if fedora failed to disable the link
local addresses. I don't know, but I suspect this is a fedora bug.
> Here's the complete list of tests failures. Since most are not new I
> have not investigated them. But we should get them fixed.
>
> - Known problem: my strongswan is out of date (where do I get an update?).
>
> - I include "wip" and "skiptest" entries.
>
> kvmplutotest netkey-audit-01 good!=bad,west:bad,east:bad
Probaly needs updated audit library that have the IKE and IPsec
categories added instead of those showing up as "UNKNOWN"
> kvmplutotest ikev1-initial-contact wip!=bad,west:missing-baseline,east:missing-baseline
I think this was just added, we never tested sending the VID. But we
don't act differently based on the VID ourselves.
> kvmplutotest ikev2-01-fallback-ikev1 good!=bad,west:bad,east:bad
Probably due to retry/retransmit output differences (eg 20s, 40s now is
500ms, 1s, 2s, etc. and for this one we cannot disable retransmits
because we need it to test the fallback.
> kvmplutotest ikev2-delete-01 good!=bad,west:bad,east:ok
> kvmplutotest ikev2-delete-02 good!=bad,west:bad,east:ok
don't know about these.
> kvmplutotest ikev2-delete-03-valgrind bad,west:missing-baseline,east:missing-baseline
lots of ephemeral data.
> kvmplutotest ikev2-delete-04 wip!=bad,west:missing-baseline,east:missing-baseline
> kvmplutotest ikev2-12-x509-ikev1 good!=bad,west:bad,east:ok
> kvmplutotest ikev2-12-x509-ikev1-rw good!=bad,west:bad,east:ok
> kvmplutotest ikev2-12-transport-psk good!=bad,west:bad,east:bad
> kvmplutotest ikev2-18-x509-alias good!=bad,east:ok,road:bad
not sure.
> kvmplutotest ikev2-19-x509-auto-start good!=bad,east:missing-baseline,road:missing-baseline
this one does not run well in automated tests.
> kvmplutotest ikev2-41-rw-replace good!=bad,east:ok,road:bad
> kvmplutotest ikev2-42-rw-replace-responder good!=bad,east:ok,road:bad
> kvmplutotest ikev2-45-impair-gx-02 good!=bad,EXPECT:west.pluto.log,west:bad,east:ok
> kvmplutotest ikev2-46-basic-psk-netkey good!=bad,west:bad,east:bad
> kvmplutotest ikev2-47-priority wip!=bad,west:missing-baseline,east:missing-baseline
> kvmplutotest ikev2-48-nat-cp good!=bad,east:ok,road:bad
> kvmplutotest ikev2-24-cryptoload wip!=bad,west:missing-baseline,east:missing-baseline
> kvmplutotest ikev2-27-uniqueid wip!=bad,east:missing-baseline,north:missing-baseline
> kvmplutotest ikev2-29-no-rekey good!=bad,west:bad,east:ok
> kvmplutotest ikev2-ddns-02 good!=bad,west:bad,east:bad
> kvmplutotest ikev2-04-basic-x509-nhelpers0 good!=bad,west:bad,east:bad
> kvmplutotest ikev2-10-2behind-nat wip!=bad,east:missing-baseline,road:missing-baseline
> kvmplutotest ikev2-algo-sha2-05 good!=bad,west:bad,east:bad
> kvmplutotest ikev2-algo-sha2-07 wip!=bad,west:missing-baseline,east:missing-baseline
> kvmplutotest ikev1-impair-gx-02 good!=bad,west:missing-baseline,east:missing-baseline
> kvmplutotest ikev2-minor-version-initiator good!=bad,west:ok,east:bad
> kvmplutotest ikev2-isakmp-reserved-flags-01 good!=bad,west:bad,east:bad
> kvmplutotest ikev2-payload-reserved-flags-01 good!=bad,west:bad,east:bad
> kvmplutotest ikev2-allow-narrow-01 good!=bad,EXPECT:west.pluto.log,west:bad,east:ok
> kvmplutotest ikev2-allow-narrow-02 good!=bad,EXPECT:west.pluto.log,west:bad,east:ok
> kvmplutotest ikev2-allow-narrow-03 good!=bad,EXPECT:west.pluto.log,west:bad,east:ok
> kvmplutotest ikev2-ddos-01 wip!=bad,east:missing-baseline,road:missing-baseline
> kvmplutotest ikev2-switchnat-01 wip!=bad,east:missing-baseline,road:missing-baseline
> kvmplutotest ikev1-switchnat-01 good!=bad,east:missing-baseline,road:missing-baseline
> kvmplutotest ikev1-connswitch-ports-01 wip!=bad,west:missing-baseline,east:missing-baseline
I'll look through your newer testrun to see what's going on with those.
> kvmplutotest nflog-02-conn good!=bad,west:bad,east:ok
> kvmplutotest nflog-03-conns good!=bad,west:bad,east:ok
> kvmplutotest ikev2-43-init-retransmit good!=bad,west:bad,east:bad
> kvmplutotest ikev2-frag-02-ipv6 good!=bad,west:bad,east:bad
> kvmplutotest newoe-01-whack good!=bad,west:bad,east:ok
> kvmplutotest newoe-02-klips wip!=bad,EXPECT:road.pluto.log,east:missing-baseline,road:missing-baseline
> kvmplutotest newoe-04-01 good!=none,NO-OUTPUT
> kvmplutotest newoe-05-hold-pass good!=bad,east:ok,road:bad
> kvmplutotest newoe-06-prio good,EXPECT:road.pluto.log
> kvmplutotest newoe-08-ike-replace-responder good!=bad,east:ok,road:bad
> kvmplutotest newoe-15-portpass good!=bad,east:ok,road:bad
> kvmplutotest newoe-16-pass-hold good!=bad,east:ok,road:bad
> kvmplutotest newoe-17-block-in-clear-clear good!=bad,east:missing-baseline,road:missing-baseline
> kvmplutotest newoe-18-poc-block good!=bad,east:ok,road:bad
> kvmplutotest newoe-18-cop-block good!=bad,east:ok,road:bad
> kvmplutotest newoe-18-private-clear good!=bad,east:bad,road:ok
> kvmplutotest newoe-18-private-block good!=bad,east:ok,road:bad
> kvmplutotest newoe-19-poc-poc-clear wip!=bad,east:missing-baseline,road:missing-baseline
I think most of these are the 8 vs 9 ping packets issue :/ I wonder if
we can tweak the parameters so we have less or no chance that this
happens.
> ctltest pluto-unit-01 good!=none,NO-OUTPUT
> ctltest pluto-unit-02 good!=none,NO-OUTPUT
> ctltest pluto-dontreky-expiry-01 good!=none,NO-OUTPUT
We haven't looked at ctltest types.
> kvmplutotest dpd-01-netkey good!=bad,EXPECT:west.pluto.log,west:bad,east:ok
> kvmplutotest dpd-04 good!=bad,west:bad,east:ok
> kvmplutotest dpd-08 wip!=bad,west:missing-baseline,east:missing-baseline
> kvmplutotest dpd-08-netkey wip!=bad,west:missing-baseline,east:missing-baseline
> kvmplutotest ikev2-liveness-06 good,EXPECT:east.pluto.log
> kvmplutotest ikev2-liveness-07 good!=bad,EXPECT:east.pluto.log,east:bad,road:bad
these also are often about timing and one or more packet losses.
> kvmplutotest delete-sa-03 wip!=bad,west:bad,east:bad
> kvmplutotest delete-state-01 good!=bad,west:bad,east:ok
We'd have to look at these, especially with the delete patch.
> kvmplutotest x509-pluto-01 good!=bad,west:bad,east:bad
> kvmplutotest x509-pluto-02 good!=bad,east:ok,north:bad
> kvmplutotest x509-pluto-03 good!=bad,west:bad,east:bad
old tests obsoletes by the nss-*-x509* tests
> kvmplutotest nat-pluto-10 good!=bad,EXPECT:east.pluto.log,east:missing-baseline,north:missing-baseline
> kvmplutotest nat-dpd-pluto-01 wip!=bad,east:missing-baseline,north:missing-baseline
> skiptest xauth-pluto-17 wip!=none,NO-OUTPUT
> kvmplutotest xauth-pluto-20 good!=none,NO-OUTPUT
> kvmplutotest aggr-pluto-03 wip!=none,NO-OUTPUT
> kvmplutotest basic-pluto-10 wip!=bad,west:missing-baseline,east:missing-baseline
> kvmplutotest basic-pluto-01-valgrind good!=bad,west:missing-baseline,east:missing-baseline
> kvmplutotest basic-pluto-11 good!=bad,west:bad,east:bad
> kvmplutotest basic-pluto-12-netkey wip!=bad,west:missing-baseline,east:missing-baseline
> kvmplutotest basic-pluto-14-klips-route wip!=bad,west:missing-baseline,east:missing-baseline
I would have to look at these.
> kvmplutotest x509-chain-01 good!=bad,west:bad,east:bad
> kvmplutotest x509-chain-02 good!=bad,west:bad,east:bad
> kvmplutotest x509-chain-03 good!=bad,west:bad,east:bad
old tests obsoletes by the nss-*-x509* tests
> kvmplutotest netkey-algo-camellia-01 good!=bad,west:bad,east:ok
> kvmplutotest algo-pluto-04 wip!=bad,west:missing-baseline,east:missing-baseline
> kvmplutotest algo-pluto-13-invalid-3des good!=bad,west:bad,east:ok
> kvmplutotest interop-ikev2-strongswan-16-ah-initiator-sha512 good!=bad,west:bad,east:ok
> kvmplutotest interop-ikev2-strongswan-17-ah-initiator-sha256 good!=bad,west:bad,east:ok
> kvmplutotest interop-ikev2-strongswan-22-cp-responder-psk good!=bad,east:ok,road:bad
> kvmplutotest interop-ikev2-strongswan-26-ke-mismatch-responder good!=bad,west:bad,east:ok
> kvmplutotest netkey-pluto-07 wip!=bad,west:bad,east:bad
> kvmplutotest labeled-ipsec-01 wip!=bad,west:missing-baseline,east:missing-baseline
not sure. partially strongswan version?
> kvmplutotest ipv6-v6-through-v6-klips-klips incomplete!=bad,west:bad,east:bad
> kvmplutotest ipv6-transport-mode-01-klips-klips incomplete!=bad,west:bad,east:bad
> kvmplutotest ipv6-transport-mode-02-netkey-netkey incomplete!=bad,west:bad,east:bad
> kvmplutotest ipv6-transport-mode-03-klips-netkey incomplete!=bad,west:bad,east:bad
> kvmplutotest ipv6-tunnel-mode-01-klips-klips incomplete!=bad,west:bad,east:bad
> kvmplutotest ipv6-tunnel-mode-02-netkey-netkey good!=bad,west:bad,east:bad
> kvmplutotest ipv6-tunnel-mode-03-klips-netkey incomplete!=bad,west:bad,east:bad
> kvmplutotest ipv6-transport-ts-mode-04-netkey-netkey incomplete!=bad,west:bad,east:bad
> kvmplutotest ipv6-tunnel-mode-03-rw wip!=bad,east:missing-baseline,road:missing-baseline
> kvmplutotest ipv6-tunnel-mode-04-rw good!=bad,east:bad,road:bad
> kvmplutotest ikev2-ipv6-transport-mode-02-netkey-netkey wip!=bad,east:missing-baseline,road:missing-baseline
I think your host needs ipv6 fixes on the bridge device?
> kvmplutotest interop-ikev2-racoon-01-noconn good!=bad,west:missing-baseline,east:missing-baseline
> kvmplutotest interop-ikev2-racoon-03-psk-initiator good!=none,NO-OUTPUT
not sure on those
> kvmplutotest interop-ikev2-racoon-04-x509-responder wip!=bad,west:missing-baseline,east:missing-baseline
> kvmplutotest interop-ikev2-racoon-05-x509-initiator good!=none,NO-OUTPUT
prob old x509 setup.
> kvmplutotest interop-ikev2-strongswan-03-psk-initiator good!=bad,west:bad,east:ok
> kvmplutotest interop-ikev2-strongswan-04-x509-responder good!=bad,west:bad,east:ok
> kvmplutotest interop-ikev2-strongswan-07-strongswan good!=bad,west:bad,east:ok
> kvmplutotest interop-ikev2-strongswan-10-nat-initiator good!=bad,east:ok,road:bad
> kvmplutotest interop-ikev2-strongswan-11-nat-initiator good!=bad,east:ok,road:bad
> kvmplutotest interop-ikev2-strongswan-13-ah-initiator good!=bad,west:bad,east:ok
> kvmplutotest interop-ikev2-strongswan-14-delete-sa good!=bad,west:bad,east:ok
> kvmplutotest interop-ikev2-strongswan-15-create_child_sa good!=bad,west:bad,east:bad
> kvmplutotest interop-ikev2-strongswan-17-delete-sa-responder good!=bad,west:bad,east:ok
> kvmplutotest interop-ikev2-strongswan-19-x509-res-certreq good!=bad,west:bad,east:bad
> kvmplutotest interop-ikev2-strongswan-27-fragmentation good!=bad,west:bad,east:ok
strongswan version
> kvmplutotest x509-pluto-frag-00 good!=bad,east:bad,road:ok
> kvmplutotest x509-pluto-frag-01 good!=bad,east:bad,road:ok
> kvmplutotest x509-pluto-frag-03 good!=bad,east:bad,road:ok
> kvmplutotest x509-pluto-frag-04 good!=bad,east:bad,road:ok
obsoleted tests.
> kvmplutotest ikeport-01 wip!=bad,west:missing-baseline,east:missing-baseline
never worked so far.
> skiptest fips-05-ikev1-gcm wip!=none,NO-OUTPUT
work in progress on booting kernel in real fips mode.
Paul