[Swan-dev] [Testing] Test Suite & Docker

Antony Antony antony at phenome.org
Sun May 15 15:51:49 UTC 2016

Hi Ondrej, 
I am still on F22:) ikev2-37-docker-rw works for me on F22. 

A quick workaround for the 0/0 problem could the patch below.
Remember the workaround will cause the test to fail for cosmetic reasons.
However you can check the actual output file to see ping etc works.

less OUTPUT/road.console.txt
diff road.console.txt OUTPUT/road.console.txt

diff --git a/testing/utils/swantest b/testing/utils/swantest
index 3e6173e..e134cd9 100755
--- a/testing/utils/swantest
+++ b/testing/utils/swantest
@@ -1735,7 +1735,7 @@ def docker_add_net_bridges(args, test, dnamei, dnamer, dnamen):
     pcap_file_ext = '.pcap'
     pcap_file = 'OUTPUT/' + 'swan12' + pcap_file_ext

-    prefix = "0/0"
+    prefix = "" # rfc5737. the real address will come with network restart
     if test["initiator"] == "west":

There is no list of dockerpluto tests. I guess it would be nice to have one, never had any interest by others. I use it manually and I can swift through the differences easily!!

Most netkey tests will run on docker with cosmetic output differences.

My understanding is klips is not namespace aware?
I don't have a detailed answer how to make protostack=klips work under docker.
The module is loaded on the host. All instances share the same module.


On Sun, May 15, 2016 at 04:21:10PM +0200, Ondrej Moris wrote:
> Hey Antony, thanks for your reply, sorry for such a delayed answer,
> please see my inline comments...
> On 05/11/2016 09:21 PM, Antony Antony wrote:
> > Hi Ondrej,
> > here is a quick response. Do you still have the system where you followed the  steps in [1]? 
> >
> > On Wed, May 11, 2016 at 01:42:37PM +0200, Ondrej Moris wrote:
> >> Hi,
> >>
> >> a few months ago I became aware of "libreswan testing suite docker
> >> adventures" [1].Then I had a chance to have a brief chat about it with
> >> Paul during his visit on DevConf in Brno, Czech Republic. Since it
> >> looked more or less like an experiment which was stalled at some point
> >> and I considered it to be very interesting idea I promised to lend a
> >> hand with it.
> >>
> >> There are two main reasons why I think it is worth resurrection:
> >>
> >> (1) It is faster and cleaner way of multi-host network testing, network
> >> namespaces represent transparent network separation (different IP
> >> stacks) and no baremetal is needed for test suite execution.
> >>
> >> (2) It will allow to run the test suite on more linux distributions.
> >> There are docker base images for Fedora, RHEL, Debian, (Open) SUSE (LE),
> >> etc.The current test suite based on KVM virtualization has an essential
> >> dependency on 9P filesystem since guests need to share testing directory
> >> with their host. However, 9P FS is not available in all kernel and qemu
> >> distributions and it is no longer maintained AFAIK. Relaxing this
> >> dependency (e.g. via NFS) seems to be too complicated and might
> >> interfere with the testing. On the other hand there is docker based on
> >> concepts of control groups and namespaces which are widespread in linux
> >> distributions for some time.
> >>
> >> Following steps in [1] test suite can be set-up easily, however it is
> >> not possible to execute any test at the moment. Obviously that part of
> >> the docker adventures are yet to happen. Moreover there are various
> >> mutually independent bits related to docker in testing directory
> >> (docker, pluto/ikev2-37-docker-rw/docker,
> >> pluto/ikev2-37-docker-rw/runme.sh and utils/swantest).
> > ignore the runme.sh. That is only for reference or to debug swantest.
> > try the steps below, which is mentioned in [1]
> >
> > cd /home/build/libreswan/
> > make programs
> > cd /home/build/libreswan/testing/pluto/ikev2-37-docker-rw
> > ../../utils/swantest --docker 
> >
> > make sure you have the latest master from github, especially 9b39035e1e9d1d7d9 (test results are updated in this commit. Results drifted away over time).
> Yes, that's something I started with at the beginning. But there seems
> to be some issue while adding bridges in docker_containers_start_stop,
> actually the problem is in docker_net_bridge, the latest pipework I have
> cannot consume 0/0 prefix:
> sudo /usr/local/bin/pipework br51789982 -i eth0 road-ikev2-37-docker-rw 0/0
> ipcalc: bad IPv4 address: 0
> Error: an inet address is expected rather than "".
> It is not surprising since ipcalc -bjust does not work with 0/0 in (at
> least in Fedora 23). At that point I mistakenly made a conclusion that
> "swantest --docker" is not finished and turned to runme.sh :-/.
> With prefix = "" ipcalc does not complain, swantest --docker
> works and the testing is not aborted. Most likely ipcalc is distribution
> specific or pipework used something else than ipcalc -b to get broadcast
> address. Anyway that does not matternow.
> Is ikev2-37-docker-rw expected to fail? I see some minor issues on
> "east" (outputs are semantically equivalent even thought they differ)
> but "road" has some real issues (ping failed, I have not time to check
> whyyet).
> >
> >> At the moment, most viable option seems to be runme.sh but it is calling
> >> non-existing guestbin/swan-docker-run. I guess swan-run can be used
> >> there with some very minor updates, is that what swan-docker-run was
> >> meant to be? 
> > as I said before to run a single test use "swantest --docker" to run mutiple tests
> > speficy the type in TESTLIST. If it is "dockerplutotest" then "make check" should run them cuncurrently. Complex bits in swan test is to run several tests cuncurrently.
> > I run single test while for quick and easy testing.
> Do you have a list of tests which fall into "dockerplutotest" category
> Antony? I would assume there is "a prototype" test ikev2-37-docker-rw at
> the moment based probably on ikev2-09-rw-rsa and there are no claims
> about the other tests yet. Or are there?
> >
> >> Clearly docker and and pluto/ikev2-37-docker-rw/dockerare just some PoC
> >> with no future progress, is that correct?
> >>
> >> Finally, there are docker test execution code in swantest but it is not
> >> used anywhere. This seems to be rather complex and I am not sure how
> >> complete it is. I guess it represents the same functionality as
> >> runme.sh, doesn't it?
> >> Clearly, the only missing step in "dockerization" of the test suite is
> >> to finish the test driver (i.e. probably having runme.sh steps covered
> >> by swantest code). Test cases are basically already both KVM and Docker
> >> friendly.
> After reading your reply I am very happy to re-phrase that paragraph -
> the test driver actually seems to be ready. Hence the next steps are
> probably to try out some good KVM pluto tests using docker, to find out
> what are the differences in their output and to sanitize them somehow.
> Is that correct?
> One more thing, one of steps in [1] forbids using KLIPS IPsec stack. I
> am aware of some potential issues and doubts about using KLIPS in LXC
> containers (so basically it applies to Docker too). On the other hand I
> am not aware of any obvious reason why clearly explained reason why (yet
> ipsec does not start with protostack=klips in the container), can you
> shed some light on this matter?
> >>
> >> So the crucial question is - are you interested in discussing future of
> >> the remaining parts of docker test suite?
> >>
> >> [1] https://libreswan.org/wiki/Test_Suite_-_Docker

More information about the Swan-dev mailing list