[Swan-dev] testing and unstable dns

[Antony Antony]

On Mon, Apr 19, 2021 at 02:02:39PM -0400, Andrew Cagney wrote:
> 
> 
> On Mon, 19 Apr 2021 at 11:53, Antony Antony <antony at phenome.org> wrote:
> 
>     On Sat, Apr 17, 2021 at 08:33:18PM -0400, Andrew Cagney wrote:
>     > BTW, I took a look at swan-prep --dnssec.  As best I can the big
>     difference
>     > between namespaces and KVM is when the config files are installed:
>     >
>     > - with KVMs the nsd and unbound directories are set up before the test is
>     run
>     > (during transmogrify)
> 
>     transmogrifying once is not a best solution for unbound and nsd.
>     some tests can modify the config files. So next test may start with a wrong
>     file.
> 
>     I think the swan-prep should to copy fresh config files every time.
> 
> 
> Moving the nsd/unbound stuff out of transmogrify makes sense.
>  
> 
> 
>     > - with namespaces, the nsd and unbound directories are set up as part of
>     some
>     > interesting mounts by swan-prep
>     >
>     > would things be more straight forward if, for namespaces, the directories
>     were
>     > set up behind the scenes before the test starts (I'm mainly thinking of
>     those
> 
>     the namespace directories and files, which are bind mount, should be setup
>     in swan-prep.  especially because we want to restart inside a vm(east or
>     west..) manually, inside a namespace, without resetting the all namespaces
>     of a test. So I think we should leave those tasks in swan-prep. It should
>     not be in namespace test runner.
> 
> 
> I don't follow.
> 
> For KVMs, runner is required to establish a minimal environment before the
> first *.sh command is run:
> - all the VMs are booted
> - at the bash prompt
> - /testing is mounted
> - CWD is the directory containing the tests
> - where applicable, libreswan is installed
> - hostname is set
> - /etc is in a state fit to be scribbled on
> (I'm sure there is other stuff)
> while this is currently implemented by walking the VM through a boot-and-login
> sequence, there's nothing to rule out using snapshots, say.  Just as long as
> the environment is established before the test starts.
> 
> If I were to type "reboot" in such a vm, then I'll need to first manually
> re-establish the above before entering the first shell command.  Why should
> namespaces be different?  If namespaces and KVM established some minimum
> environment before running tests then I think the odds of tests runing under
> both frameworks would be greatly improved.
> 
> BTW, I'd take the above list as a starting point for discussion.  Currently
> swan-prep has to deal with cleaning up from previous tests, I think that's a
> bug.

Hi Andrew,

I have been trying to reply to this e-mail for two weeks. This is the best 
I could come with on your discovery of the clean-up feature as a bug. In my 
opinion, it is a wrong conclusion!

Moreover your conclusion makes me feel how different our worldviews are when 
it comes to testing scripts and clean-up features! I hope you did not remove 
those yet! I haven't looked since this e-mail, a bit scared to imagining I 
would have to fight it.

I personally see test restarting (interactively running one script from test
dir, attaching gdb...), while developing, as the first reason for the
testing environment to exist! Testrun is a primary feature we as group, but 
secondary for me:) It would be sad to see the restart feature being chipped 
away on your conclusion that it is a bug.

I have been scratching my head to understand how you would reach the 
conclusion it is a bug. As far as I can remember it has been there from the 
beginning of KVM testing scripts, such as swan-prep.

To be clear do not remove the hooks put in there to support rerunning
a test manually by script by script without using runner: such as swan-prep 
cleaning up previous instances,
also in some init scripts, westinit/roadinit.sh, may remove the 
addressees/routes added during a run that is to clean up the previous run.  
e.g MOBIKE dns. I don"t agree with removing those even if it may speedup a
test run. It is an essential feature! And rebooting KVM is not an 
alternative!

If you feel avoiding clean-up would considerably speed up our testrun feel 
free to add some options which by default disabled and could be optionally 
enabled on testrun say on testing.libreswan.org.

regards,
-antony