[Swan-dev] testing and unstable dns

[Paul Wouters]

On Mon, 19 Apr 2021, Andrew Cagney wrote:

> Moving the nsd/unbound stuff out of transmogrify makes sense.

It would be nice if we could also start them manually and specifying the
config file, so we don't need as many bind mounts and things.

>       > - with namespaces, the nsd and unbound directories are set up as part of some
>       > interesting mounts by swan-prep
>       >
>       > would things be more straight forward if, for namespaces, the directories were
>       > set up behind the scenes before the test starts (I'm mainly thinking of those
>
>       the namespace directories and files, which are bind mount, should be setup
>       in swan-prep.  especially because we want to restart inside a vm(east or
>       west..) manually, inside a namespace, without resetting the all namespaces
>       of a test. So I think we should leave those tasks in swan-prep. It should
>       not be in namespace test runner.
> 
> 
> I don't follow.
> 
> For KVMs, runner is required to establish a minimal environment before the first *.sh command is run:
> - all the VMs are booted
> - at the bash prompt
> - /testing is mounted
> - CWD is the directory containing the tests
> - where applicable, libreswan is installed
> - hostname is set
> - /etc is in a state fit to be scribbled on
> (I'm sure there is other stuff)
> while this is currently implemented by walking the VM through a boot-and-login sequence, there's nothing to rule out using snapshots, say.  Just as long as the
> environment is established before the test starts.

I'm a bit nervous about snapshots. We originally went with reboot uml/kvm
between each test to ensure a clean slate. If we start re-using snapshots,
I fear there will be secret sauces in these snapshots. While a base
snapshot for the entire test run seems a good idea, having them per-test
seems like a bad idea.

swan-prep ensures there are no leftovers of previous test. This helps us
when 1 test breaks something, so that not all subsequent tests fail. Eg
because there is an additional certificate in NSS or something.

> If I were to type "reboot" in such a vm, then I'll need to first manually re-establish the above before entering the first shell command.  Why should
> namespaces be different?  If namespaces and KVM established some minimum environment before running tests then I think the odds of tests runing under both
> frameworks would be greatly improved.

Well, you cannot reboot a namespace :P

> BTW, I'd take the above list as a starting point for discussion.  Currently swan-prep has to deal with cleaning up from previous tests, I think that's a bug.

Defense in depth for 1 failure to cleanup messing up 500 test results.

>       I feel it would be sad to see if you move swan-prep into several shell
>       scripts, instead of fixing swan-prep.

whether swan-prep is one script or many doesn't matter too much to me.
As long as it remains 1 line in the scripts to run.

Note that things have been breaking for me lately too. x509 tests in
namespaces no longer work because nss complains about importing
duplicates - although i think this is actually an nss bug.

It would be good if we can move testing to use /var/lib/ipsec/nss
because right now we are fighting between that and /etc/ipsec.d
and if you want to test an rpm install it gets weird.

The NS directories end up root owned and cannot be deleted. I also
suspect they are accidentally re-used at times. The test should wipe
these at the end of the test (if --shutdown was given)

Paul