[Swan-dev] test suite status

Antony Antony antony at phenome.org
Sun Jun 18 16:56:00 UTC 2017


On Sun, Jun 18, 2017 at 12:19:25PM -0400, Paul Wouters wrote:
> On Sun, 18 Jun 2017, D. Hugh Redelmeier wrote:
> 
> > After a pause of a few months, I ran the test suite last night.
> > I tested HEAD, as of ce5d67b98214746e8e55a2a1c401343117dba1aa.
> > 
> > A *lot* of tests seem to have failed.  I looked at a couple of failures
> > and got bored.  They were just random changes in XFRM logging and I had no
> > way of knowing if they actually mattered.
> > 
> > Are these results expected?
> 
> You caught me in the middle of a big update. 
>
> The problem was that the "ip xfrm pol" output had two issues:
> 
> 1) pseudo random order of in/out/fwd entries in "ip xfrm pol" output
> 2) Spurious extra 0.0.0.0/0 socket options line.

this sounds like an iprovement.
Is the state order is important? Isn't how how packet processing will find the state.

> To fix these, I updated ipsec look to use the one line ip xfrm pol
> output using -o, then sort these, and then expand them again to
> multiline for readability. I also filter the 0.0.0.0 socket lines out
> before expanding.
> 
> So while 2) was easy to predict and edit before the run, I hadn't done
> that for issue 1), so once the current testrun with these changes
> has finished, I will grab the diffs related to ip xfrm pol, and update
> them. This test run is still going at:

I wish "ipsec look" output change and the test output change was pushed together.
For example you could run tests on a branch and merge them together.

It could have avoided others stumbling on lots of failed test results on a Sunday morning and waiting for you  to push "fix":) Just a wishful thinking. We will probably break something else.


More information about the Swan-dev mailing list