[Swan-dev] having failing testscripts fail tests
Andrew Cagney
andrew.cagney at gmail.com
Tue Mar 24 16:13:00 EET 2015
One thing I've noticed with tests is that they can end up spending a lot of
time trying to push a train wreck up hill.
For instance, looking in basic-pluto-01/westinit.sh:
/testing/guestbin/swan-prep
# confirm that the network is alive
ping -n -c 4 -I 192.0.1.254 192.0.2.254
# make sure that clear text does not get through
iptables -A INPUT -i eth1 -s 192.0.2.0/24 -j LOGDROP
# confirm with a ping
ping -n -c 4 -I 192.0.1.254 192.0.2.254
ipsec setup start
/testing/pluto/bin/wait-until-pluto-started
ipsec auto --add westnet-eastnet
ipsec auto --status
echo "initdone"
should either swan-prep or wait-until-pluto-started fail then there really
is no point in continuing with the test; yet we do.
The obvious fix is to tweak the code feeding these commands to the remote
end so that it checks the exit code. Something like:
test_domain.domain.log("[%s]# %s", basename, line)
status = test_domain.shell.run(line, timeout=TEST_TIMEOUT)
if status:
# XXX: Can't abort as some ping commands are
# expected to fail.
test_domain.domain.log("command '%s' failed with status
%d", line, status)
as the comment notes, there are some got-yas:
- some commands are expected to fail; we'd need to tweak them (for instance
changing some "ping ..." commands to "! ping ...")
- sometimes things just fail (again pings), yet the test stumbles on and
the rest seems to pass; a retry should flush this out
thoughts?
Andrew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libreswan.org/pipermail/swan-dev/attachments/20150324/2fa322fe/attachment.html>
More information about the Swan-dev
mailing list