[Swan-dev] drop ipsec-auto-up.n.sed
andrew.cagney at gmail.com
Tue Sep 29 02:49:47 UTC 2020
On Mon, 28 Sep 2020 at 21:53, Paul Wouters <paul at nohats.ca> wrote:
> On Mon, 28 Sep 2020, Andrew Cagney wrote:
> > I'm planning on removing the sanitizer ipsec-auto-up.n.sed.
> I understand why, but I also understand's Antony's point.
> To make it harder, once I rewrite the tests to also be documentation,
> we have an additionally issue of adding weird things to the configs
> people might copy. Like slow retransmit timeouts or impairs.
slow-retransmits should be deleted (I thought I had, I see I hadn't)
Your suspicion that IKEv1 didn't suppress retransmits turned out to be
because the tests in question weren't trying to supress retransmits.
> I'm also a bit nervous if 90% of our tests use --impair
> delete-on-retransmit At one point are we still testing "regular operation"
> code with our tests.
Two types of tests:
1. regression tests - these should be deterministic - someone reports a bug
or we add a feature and we add a test for it
2. stress testing; the whole point of these is to be non-deterministic
but we're discussing deterministic tests here
> Antony's filter in ipsec-auto-up.n.sed does address most of these
This assertion is not true. Please look at the examples I posted. It
silently removes important contextual information from test output - namely
that a retransmit occurred - while leaving behind the consequences. The
upshot is that a retransmit still causes the test to fail; only that the
retransmit happened is obscured.
With the sanitizer disabled I got roughly the same number of fails.
> But other issues still remain, such as retransmits causing
> different traffic counters in the output and causing false positives.
> So, I guess I have no good answers or strong opinions here. Just
> concerns :)
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Swan-dev