[Swan-dev] swantest fixes + --include <regex> --exclude <regex> --testingdir <dir>

[Andrew Cagney]

I drained my makefiles branch.  For swantest it contains the following
fixes:

- if there doesn't seem to be a directory for a test it skips it, logging
an error
A test without a directory is the root cause of one of the recent swantest
crashes.

- if a test has already been run, then it logs a warning that the test will
be skipped
While the logic around retrying and skipping previous tests still seems
sketchy (trace how output_dir gets modified), at least it isn't quietly
skipping stuff for no apparent reason

- a very very old log call is (hopefully) fixed
I fixed the root cause so the code isn't being executed :-/

In addition, there are changes to add the following options:

--include <regex> (not so well tested)
--exclude <regex> (in use)

These options let you filter which tests are run.  For instance:
    --exclude 'wip|bad|incomplete'
will filter out test that match that pattern.  Conversely --include should
to the reverse and only include tests that match, for instance:
    --include 'ikev2'
to just run ikev2 tests.

Implementation note.  It' is using python's un-anchored pattern match
(re.search()) against all the fields of TESTLIST so is very liberal.

--testingdir <directory> (experimental)

Instead of /testing, use <directory> as the KVM's testing directory.  While
this technically works, the actual tests may not cooperate.

Andrew

PS: One of my runs testing this deadlocked with a backtrace.  Guess I get
to look at that :-(
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libreswan.org/pipermail/swan-dev/attachments/20150306/cfc39502/attachment.html>