[Swan-dev] test suite status

[Paul Wouters]

On Sun, 18 Jun 2017, D. Hugh Redelmeier wrote:

> After a pause of a few months, I ran the test suite last night.
> I tested HEAD, as of ce5d67b98214746e8e55a2a1c401343117dba1aa.
>
> A *lot* of tests seem to have failed.  I looked at a couple of failures
> and got bored.  They were just random changes in XFRM logging and I had no
> way of knowing if they actually mattered.
>
> Are these results expected?

You caught me in the middle of a big update.

The problem was that the "ip xfrm pol" output had two issues:

1) pseudo random order of in/out/fwd entries in "ip xfrm pol" output
2) Spurious extra 0.0.0.0/0 socket options line.

To fix these, I updated ipsec look to use the one line ip xfrm pol
output using -o, then sort these, and then expand them again to
multiline for readability. I also filter the 0.0.0.0 socket lines out
before expanding.

So while 2) was easy to predict and edit before the run, I hadn't done
that for issue 1), so once the current testrun with these changes
has finished, I will grab the diffs related to ip xfrm pol, and update
them. This test run is still going at:

http://testing.libreswan.org/results/testing/v3.20-603-gce5d67b-master

Paul