[Swan-dev] two test runs, on different machines, matched
Andrew Cagney
andrew.cagney at gmail.com
Sun Sep 26 20:10:45 UTC 2021
total: 967
total/failed: 29
total/ignored: 91
total/passed: 847
ikev2-32-nat-rw-rekey failed east:output-different road:output-different
ikev2-48-nat-cp-start failed road:output-different
newoe-20-ipv6 failed east:output-different road:output-different
newoe-27-replace-sa-auth-authnull failed road:output-different
certoe-12-nat-server failed road:output-different
certoe-14-poc-del-slash32 failed east:EXPECTATION
certoe-15-poc-east-west failed east:EXPECTATION
certoe-17-asymmetric-cert-nat-testing-client-crash failed
east:output-different road:output-different
certoe-18-pass-then-go-slash32-keyingtries1 failed road:output-different
newoe-18-poc-cop-port22-both-reorder failed east:output-different
road:output-different
dpd-01 failed west:output-different
dpd-02 failed east:output-different
ikev2-liveness-05 failed west:output-different
ikev2-liveness-06 failed east:output-different road:output-different
ikev2-liveness-07 failed road:output-different
ikev2-child-rekey-06-deadlock failed west:output-different
ikev2-child-rekey-09-windows failed road:output-different
delete-sa-05 failed west:output-different
ikev1-rw-multiple-subnets failed east:output-different
ikev1-rw-multiple-subnets-split failed east:output-different
road:output-different
ikev2-rw-multiple-subnets failed east:output-different
ikev2-rw-multiple-subnets-2 failed east:output-different
ikev2-rw-multiple-subnets-3 failed east:output-different
ikev2-rw-multiple-subnets-4-mismatch failed east:output-different
road:output-different
ikev2-labeled-ipsec-04-no-label-on-initiator failed east:output-different
nss-cert-crl-03-strict failed east:output-different
ikev2-mobike-03 failed road:EXPECTATION
ikev2-systemrole-03-certs failed west:output-different
ikev2-crossing-streams-03-iface failed east:output-different
road:output-different
On Thu, 9 Sept 2021 at 20:41, Andrew Cagney <andrew.cagney at gmail.com> wrote:
>
>
> On Sun, 5 Sept 2021 at 11:17, Paul Wouters <paul at nohats.ca> wrote:
>
>> On Sun, 5 Sep 2021, Andrew Cagney wrote:
>>
>> > On Sun, 5 Sept 2021 at 09:44, D. Hugh Redelmeier <hugh at mimosa.com>
>> wrote:
>> > I ran the tests suite last night on two machines. It used to be
>> that one
>> > would expect different results from two runs "due to
>> non-determanism",
>> > even on the same machine.
>> >
>> > I got the same results on two machines. That's either good news
>> or luck.
>> > This is probably the result of hard work: thanks!
>> >
>> >
>> > It's a bit of both, for instance, these two runs had 36 failures:
>> >
>> https://testing.libreswan.org/?run=v4.5-36-gcb013f4154-main&run=v4.5-35-g6a4754e1fd-main
>> > But yes, things are definitely better.
>>
>> I think Andrew contributed most of this part in the last two months
>> actually. But good to
>> see indeed! Next up, fixing liveness/dpd and WIP tests :)
>>
>
> Time to move the tests that won't pass to wip?
>
>
>> Paul
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libreswan.org/pipermail/swan-dev/attachments/20210926/6a704bb0/attachment.html>
More information about the Swan-dev
mailing list