<div dir="ltr">  total: 967<br>  total/failed: 29<br>  total/ignored: 91<br>  total/passed: 847<br>ikev2-32-nat-rw-rekey failed east:output-different road:output-different<br>ikev2-48-nat-cp-start failed road:output-different<br>newoe-20-ipv6 failed east:output-different road:output-different<br>newoe-27-replace-sa-auth-authnull failed road:output-different<br>certoe-12-nat-server failed road:output-different<br>certoe-14-poc-del-slash32 failed east:EXPECTATION<br>certoe-15-poc-east-west failed east:EXPECTATION<br>certoe-17-asymmetric-cert-nat-testing-client-crash failed east:output-different road:output-different<br>certoe-18-pass-then-go-slash32-keyingtries1 failed road:output-different<br>newoe-18-poc-cop-port22-both-reorder failed east:output-different road:output-different<br>dpd-01 failed west:output-different<br>dpd-02 failed east:output-different<br>ikev2-liveness-05 failed west:output-different<br>ikev2-liveness-06 failed east:output-different road:output-different<br>ikev2-liveness-07 failed road:output-different<br>ikev2-child-rekey-06-deadlock failed west:output-different<br>ikev2-child-rekey-09-windows failed road:output-different<br>delete-sa-05 failed west:output-different<br>ikev1-rw-multiple-subnets failed east:output-different<br>ikev1-rw-multiple-subnets-split failed east:output-different road:output-different<br>ikev2-rw-multiple-subnets failed east:output-different<br>ikev2-rw-multiple-subnets-2 failed east:output-different<br>ikev2-rw-multiple-subnets-3 failed east:output-different<br>ikev2-rw-multiple-subnets-4-mismatch failed east:output-different road:output-different<br>ikev2-labeled-ipsec-04-no-label-on-initiator failed east:output-different<br>nss-cert-crl-03-strict failed east:output-different<br>ikev2-mobike-03 failed road:EXPECTATION<br>ikev2-systemrole-03-certs failed west:output-different<br>ikev2-crossing-streams-03-iface failed east:output-different road:output-different<br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, 9 Sept 2021 at 20:41, Andrew Cagney <<a href="mailto:andrew.cagney@gmail.com">andrew.cagney@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, 5 Sept 2021 at 11:17, Paul Wouters <<a href="mailto:paul@nohats.ca" target="_blank">paul@nohats.ca</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Sun, 5 Sep 2021, Andrew Cagney wrote:<br>
<br>
> On Sun, 5 Sept 2021 at 09:44, D. Hugh Redelmeier <<a href="mailto:hugh@mimosa.com" target="_blank">hugh@mimosa.com</a>> wrote:<br>
>       I ran the tests suite last night on two machines. It used to be that one<br>
>       would expect different results from two runs "due to non-determanism",<br>
>       even on the same machine.<br>
><br>
>       I got the same results on two machines.  That's either good news or luck. <br>
>       This is probably the result of hard work: thanks!<br>
> <br>
> <br>
> It's a bit of both, for instance, these two runs had 36 failures:<br>
>   <a href="https://testing.libreswan.org/?run=v4.5-36-gcb013f4154-main&run=v4.5-35-g6a4754e1fd-main" rel="noreferrer" target="_blank">https://testing.libreswan.org/?run=v4.5-36-gcb013f4154-main&run=v4.5-35-g6a4754e1fd-main</a><br>
> But yes, things are definitely better.<br>
<br>
I think Andrew contributed most of this part in the last two months actually. But good to<br>
see indeed! Next up, fixing liveness/dpd and WIP tests :)<br></blockquote><div><br></div><div> Time to move the tests that won't pass to wip?</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Paul<br>
</blockquote></div></div>
</blockquote></div>