[Swan-dev] Running tests in parallel using "make kvm-test"

[Andrew Cagney]

FYI,

I've added a new section to the testing page discussing how "make
kvm-test" (which uses kvmrunner.py) can be configured to run tests and
testsuites in parallel.
  https://libreswan.org/wiki/Test_Suite#Speeding_up_.22make_kvm-test.22_by_running_things_in_parallel
scroll down to see some pretty graphs :-)

Interestingly, domain reboots add so much load to the system that
trying to speed things up by running them in parallel doesn't actually
help.  Far better to instead run tests in parallel.  If we could
switch to a Linux variant that rebooted in 1-2 seconds instead of
10-20 we'll get a bigger win.

For docker, it will be interesting to see what the overhead of
creating the domains is, hopefully it won't need to be throttled.

For reference, I recommend two configurations:


- for a dedicated test server only running one testsuite

KVM_WORKERS=#cores/2
KVM_PREFIX=p1. p2. ... p#cores.


- for an interactive system (i.e., laptop, desktop) where multiple
testsuites could be running in parallel, and the machine isn't so
beefy

-- for the development checkout where quick single test turn-around is the goal

KVM_WORKERS=2  (assuming you've at least 4 cores, otherwise leave it unchanged)
leave KVM_PREFIX unchanged

--- for the baseline checkout where quicker testsuite turnaround is
the goal; but with out bogging down the entire system

leave KVM_WORKERS unchanged
KVM_PREFIX=p1. p2.


Andrew