[Swan-dev] two test runs, on different machines, matched

D. Hugh Redelmeier hugh at mimosa.com
Sun Sep 5 13:43:58 UTC 2021


I ran the tests suite last night on two machines. It used to be that one 
would expect different results from two runs "due to non-determanism", 
even on the same machine.

I got the same results on two machines.  That's either good news or luck.  
This is probably the result of hard work: thanks!

kvmrunner 6:22:43.06:   stats/skiptest/ignored/status!=good: 9
kvmrunner 6:22:43.06:   stats/wip/ignored/status!=good: 79
kvmrunner 6:22:43.06:   tests/good/errors/EXPECTATION: 3
kvmrunner 6:22:43.06:   tests/good/errors/output-different: 43
kvmrunner 6:22:43.06:   tests/good/results/failed: 46
kvmrunner 6:22:43.06:   tests/good/results/passed: 832
kvmrunner 6:22:43.06:   tests/skiptest/results/untested: 9
kvmrunner 6:22:43.06:   tests/wip/results/untested: 79
kvmrunner 6:22:43.06:   total: 966
kvmrunner 6:22:43.06:   total/failed: 46
kvmrunner 6:22:43.06:   total/ignored: 88
kvmrunner 6:22:43.06:   total/passed: 832

The machines are essentially identical except that one has an 8G of RAM 
and an SSD and the other has 16G of RAM and a HDD.  The different disk 
speeds might tend to amplify non-determinism.  The run times were 5:34 and 
6:23.


More information about the Swan-dev mailing list