[Swan-dev] testing yesterday's tree

[Andrew Cagney]

On 14 May 2018 at 15:30, D. Hugh Redelmeier <hugh at mimosa.com> wrote:
> I ran the tests on my test system.  This is the first time I've used
> Fedora 28 as the host.
>
> The tree was master.  The last commit was
> 48b79c49c080cf8c4aca63872e6290355d21ca50 (Merge branch
> 'testing-fedora28').  Thanks, Antony, for making Fedora 28 work just
> in time for me.
>
> I have appended the summary produced by kvmrunner, with the boring
> prefix removed.
>
> A lot of tests were retried (632).  What does that mean?  Why so many?

If there's a test result from a previous run then the test run is
considered a 'retry'.  Here's the breakdown:

>   ignored: 93
>   tests: 641
>   tests/failed: 81
>   tests/passed: 554
>   tests/retry: 632
>   tests/try: 9
>   tests/unresolved: 6
>   total: 734

Hopefully tries+retries == failed+passed+unresolved == 641 and that
+ignored = 734

81 failures matches testing.libreswan.org so it is a good baseline.
However, 6 unresolved is a worry.  II suspect your domains are dying,
do: make kvm-clean

To decode this dump of an internal table:

> final test results:
>   status/bad/ignored/status!=good: 1
>   status/good/failed/previous=failed: 56
>   status/good/failed/previous=passed: 22

22 tests went from pass to fail

>   status/good/passed/previous=failed: 32

and 32 went from fail to pass

>   status/good/passed/previous=passed: 513
>   status/good/passed/previous=unresolved: 3

and 6 are, worryingly unresolved

>   status/good/unresolved/previous=failed: 2
>   status/good/unresolved/previous=passed: 4
>   status/skiptest/ignored/status!=good: 21
>   status/wip/ignored/status!=good: 71



>   total: 734
>   total/failed: 81
>   total/failed/good: 81
>   total/failed/good/output-different: 81
>   total/ignored: 93
>   total/passed: 554
>   total/passed/good: 554
>   total/unresolved: 6
>   total/unresolved/good: 6
>   total/unresolved/good/output-missing: 6
> _______________________________________________
> Swan-dev mailing list
> Swan-dev at lists.libreswan.org
> https://lists.libreswan.org/mailman/listinfo/swan-dev