[Swan-dev] [Testing] Test Suite & Docker

[Ondrej Moris]

Hey Antony, thanks for your reply, sorry for such a delayed answer,
please see my inline comments...

On 05/11/2016 09:21 PM, Antony Antony wrote:
> Hi Ondrej,
> here is a quick response. Do you still have the system where you followed the  steps in [1]? 
>
> On Wed, May 11, 2016 at 01:42:37PM +0200, Ondrej Moris wrote:
>> Hi,
>>
>> a few months ago I became aware of "libreswan testing suite docker
>> adventures" [1].Then I had a chance to have a brief chat about it with
>> Paul during his visit on DevConf in Brno, Czech Republic. Since it
>> looked more or less like an experiment which was stalled at some point
>> and I considered it to be very interesting idea I promised to lend a
>> hand with it.
>>
>> There are two main reasons why I think it is worth resurrection:
>>
>> (1) It is faster and cleaner way of multi-host network testing, network
>> namespaces represent transparent network separation (different IP
>> stacks) and no baremetal is needed for test suite execution.
>>
>> (2) It will allow to run the test suite on more linux distributions.
>> There are docker base images for Fedora, RHEL, Debian, (Open) SUSE (LE),
>> etc.The current test suite based on KVM virtualization has an essential
>> dependency on 9P filesystem since guests need to share testing directory
>> with their host. However, 9P FS is not available in all kernel and qemu
>> distributions and it is no longer maintained AFAIK. Relaxing this
>> dependency (e.g. via NFS) seems to be too complicated and might
>> interfere with the testing. On the other hand there is docker based on
>> concepts of control groups and namespaces which are widespread in linux
>> distributions for some time.
>>
>> Following steps in [1] test suite can be set-up easily, however it is
>> not possible to execute any test at the moment. Obviously that part of
>> the docker adventures are yet to happen. Moreover there are various
>> mutually independent bits related to docker in testing directory
>> (docker, pluto/ikev2-37-docker-rw/docker,
>> pluto/ikev2-37-docker-rw/runme.sh and utils/swantest).
> ignore the runme.sh. That is only for reference or to debug swantest.
> try the steps below, which is mentioned in [1]
>
> cd /home/build/libreswan/
> make programs
> cd /home/build/libreswan/testing/pluto/ikev2-37-docker-rw
> ../../utils/swantest --docker 
>
> make sure you have the latest master from github, especially 9b39035e1e9d1d7d9 (test results are updated in this commit. Results drifted away over time).
Yes, that's something I started with at the beginning. But there seems
to be some issue while adding bridges in docker_containers_start_stop,
actually the problem is in docker_net_bridge, the latest pipework I have
cannot consume 0/0 prefix:

sudo /usr/local/bin/pipework br51789982 -i eth0 road-ikev2-37-docker-rw 0/0
ipcalc: bad IPv4 address: 0
Error: an inet address is expected rather than "".

It is not surprising since ipcalc -bjust does not work with 0/0 in (at
least in Fedora 23). At that point I mistakenly made a conclusion that
"swantest --docker" is not finished and turned to runme.sh :-/.

With prefix = "0.0.0.0/0" ipcalc does not complain, swantest --docker
works and the testing is not aborted. Most likely ipcalc is distribution
specific or pipework used something else than ipcalc -b to get broadcast
address. Anyway that does not matternow.

Is ikev2-37-docker-rw expected to fail? I see some minor issues on
"east" (outputs are semantically equivalent even thought they differ)
but "road" has some real issues (ping failed, I have not time to check
whyyet).
>
>> At the moment, most viable option seems to be runme.sh but it is calling
>> non-existing guestbin/swan-docker-run. I guess swan-run can be used
>> there with some very minor updates, is that what swan-docker-run was
>> meant to be? 
> as I said before to run a single test use "swantest --docker" to run mutiple tests
> speficy the type in TESTLIST. If it is "dockerplutotest" then "make check" should run them cuncurrently. Complex bits in swan test is to run several tests cuncurrently.
> I run single test while for quick and easy testing.
Do you have a list of tests which fall into "dockerplutotest" category
Antony? I would assume there is "a prototype" test ikev2-37-docker-rw at
the moment based probably on ikev2-09-rw-rsa and there are no claims
about the other tests yet. Or are there?
>
>> Clearly docker and and pluto/ikev2-37-docker-rw/dockerare just some PoC
>> with no future progress, is that correct?
>>
>> Finally, there are docker test execution code in swantest but it is not
>> used anywhere. This seems to be rather complex and I am not sure how
>> complete it is. I guess it represents the same functionality as
>> runme.sh, doesn't it?
>> Clearly, the only missing step in "dockerization" of the test suite is
>> to finish the test driver (i.e. probably having runme.sh steps covered
>> by swantest code). Test cases are basically already both KVM and Docker
>> friendly.
After reading your reply I am very happy to re-phrase that paragraph -
the test driver actually seems to be ready. Hence the next steps are
probably to try out some good KVM pluto tests using docker, to find out
what are the differences in their output and to sanitize them somehow.
Is that correct?

One more thing, one of steps in [1] forbids using KLIPS IPsec stack. I
am aware of some potential issues and doubts about using KLIPS in LXC
containers (so basically it applies to Docker too). On the other hand I
am not aware of any obvious reason why clearly explained reason why (yet
ipsec does not start with protostack=klips in the container), can you
shed some light on this matter?
>>
>> So the crucial question is - are you interested in discussing future of
>> the remaining parts of docker test suite?
>>
>> [1] https://libreswan.org/wiki/Test_Suite_-_Docker