[Swan-dev] Docker test suite

[Kim B. Heino]

(This is partly followup to Hugh's "test failures" email.)

I've been playing with Docker lately, especially how to run swan
tests on it. This is just a proof of concept and request for comments.
Please do:

ssh build at swantest.libreswan.fi
cd docker
less compile.sh

Part 1: Pull pre-built Fedora and swan-deps from docker hub. This takes
few minutes (5m on swantest) but you have to do this only twice a year.
It's commented out for this reason.

Part 2: Cleanup previous test run results, compile latest libreswan and
create new docker container from it.

Part 3: Generate new host keys (east and west) and prepare config
directories. It's easy to add road1, road2, north, south, ... hosts too.

Parts 2+3 takes about one minute on swantest and can be mostly re-used
between different tests.

Part 4: Start east and west containers and run one basic connectivity
test on them. This takes about 10 seconds.

If you want to, you can run "./compile.sh" and see it yourself. Also
take a look at tmp/{east,west,logs} files.

Some pros and cons compared to current test suite:

+ Easy to setup, requires "dnf install docker", five files in docker/*
  plus libreswan source tree. Try it (remember part 1). compile.sh runs
  in 35s on my laptop.

+ Fast. One minute plus 10-30s per basic test on swantest.

- Everything (klips...) can't be tested on Docker. This is meant for
  "in addition to", not "full replacement".

+ Tests functionality (ping or similar), not output changes.
  Output changes in the pluto code does not break tests.

- Tests functionality, something can be missed?

+ Output changes can also be checked (tmp/logs/*) if so desired.

+ Multiple tests can be run on parallel.

Things to do:

- Some real testsuite-runner and per-test config format instead of
  compile.sh and test.sh. I'm willing to write these in Python.

What do you think? Is this something that could be used on swan
testing? Should I continue from here, or forget this?