[Swan-dev] [Testing] Libreswan Testing Future - Call Notes (from May 16)

[Neal P. Murphy]

On Wed, 25 May 2016 14:25:39 +0200
Ondrej Moris <omoris at redhat.com> wrote:

> On 05/18/2016 03:40 PM, Ondrej Moris wrote:
> > 
> > Testing with KVM
> > ----------------
> > KVM is now used to emulate network under test. This approach is slow
> > because of various reasons (not only related to KVM): slow boot of
> > guests, sanity checks (eg. checking that something is not reachable
> > means waiting for timeout), transmogrification slow-down, reboots of
> > guests between subsequent tests need to clean-up network stacks. KVM
> > approach does not scale well, it works fine for 1-3 network nodes but
> > fails badly for 100. Parallel test execution does not fit into KVM by
> > the same principle. The only non-network shared directory for KVM is
> > represented by 9P filesystem which is rather slow, seemingly not
> > maintained anymore and missing completely in RHEL. Avoiding 9P means
> > using network between host and guests (eg. NFS) which might be fragile
> > and potential source of problems. Last but no least, KVM seems to losing
> > drive.
> > 
> 
> I dug into 9P replacement a bit more during the last days trying to find
> a way how to avoid network-related solution. I got an idea to use some
> smarter filesystem with separate journals and simultaneous mounts and
> tried gfs2. It is widely supported in linux distributions. I played with
> it for a while and it is possible to create gfs2 filesystem in a file on
> host, mount it via loopback device and add it to guests as a shared raw
> storage. Guests then mount it as gfs2, whoever mounts it has its own
> journal, no network is involved, but... apparently a cluster is needed
> to make gfs2 work and deploying a cluster is overcomplicated, number of
> nodes is very limited (<=16) and it needs network in the first place.
> 
> There are no other non-network options. We can try to replace 9P by NFS
> now. I will be traveling until June 7, but one of my colleagues will be
> working on that. Then we can focus on KVM performance fine-tuning of
> issues Antony described earlier in this thread. And I will get back to
> Docker afterwards.

There is one possible solution. Have all VMs mount the virtual FS of tests read-only. Then either give each VM its own writable virtual FS to store results, or have each VM store results in a RAMFS and, when done, tar/cpio that RAMFS into an unformatted (raw) COW virtual drive. The host would copy/move the results from those virtual drives to permanent storage.

It might even be possible to create a virtual FS for each test case, and hot-plug each, in turn, into a waiting idle test VM and mark the VM busy. The VM runs the test, then signals the host (somehow) that it's done. The host unplugs the virtual drive, stores the test result in the background, and marks that VM idle.

N