[Swan-dev] tests with empty expected console output

[Andrew Cagney]

Hi,

I noticed that a number of tests have empty (rather than missing)
expected console output vis:

$ find testing/pluto/ -type f -name '*.console.txt' -size 0 -print |
cut -d/ -f3 | while read t ; do grep $t testing/pluto/TESTLIST || echo
$t ; done
kvmplutotest    basic-pluto-01-valgrind    good
kvmplutotest    nat-pluto-10        good
kvmplutotest    ikev2-24-cryptoload            wip
kvmplutotest    ikev2-24-cryptoload            wip
kvmplutotest    ikev2-27-uniqueid            wip
kvmplutotest    ikev2-27-uniqueid            wip
kvmplutotest    basic-pluto-14-klips-route wip
kvmplutotest    basic-pluto-14-klips-route wip
ikev2-34-lxc
ikev2-34-lxc
kvmplutotest    nat-dpd-pluto-01    wip
kvmplutotest    nat-dpd-pluto-01    wip
kvmplutotest    basic-pluto-12-netkey    wip
kvmplutotest    basic-pluto-12-netkey    wip
kvmplutotest    ikev2-algo-sha2-07            wip
kvmplutotest    ikev2-algo-sha2-07            wip
kvmplutotest    ipv6-tunnel-mode-03-rw            wip
kvmplutotest    ipv6-tunnel-mode-03-rw            wip

Since doing this guarantees that the test will fail I'm wondering what
the logic behind this is; especially in the case of "good" tests.

>From my pov I think it would be better to be consistent and have no output vis:

$ ./testing/utils/kvmresults.py --quick testing/pluto/ | grep output-unchecked
testing/pluto/ikev2-dcookie-02 passed east:output-unchecked
west:output-unchecked
testing/pluto/ikev2-dcookie-03 passed east:output-unchecked
west:output-unchecked
testing/pluto/ikev2-19-x509-auto-start passed east:output-unchecked
road:output-unchecked
testing/pluto/ikev2-algo-esn-02 passed east:output-unchecked
west:output-unchecked
testing/pluto/ikev1-impair-gx-02 passed east:output-unchecked
west:output-unchecked
testing/pluto/ikev1-switchnat-01 passed east:output-unchecked
road:output-unchecked
testing/pluto/newoe-17-block-in-clear-clear passed
east:output-unchecked road:output-unchecked
testing/pluto/nat-pluto-10 failed east:output-unchecked north:output-different
testing/pluto/basic-pluto-01-valgrind failed east:output-unchecked
west:output-different
testing/pluto/interop-ikev2-racoon-01-noconn passed
east:output-unchecked west:output-unchecked
testing/pluto/interop-ikev2-racoon-03-psk-initiator passed
east:output-unchecked west:output-unchecked
testing/pluto/interop-ikev2-racoon-05-x509-initiator passed
east:output-unchecked west:output-unchecked