[Swan-dev] drop ipsec-auto-up.n.sed

[Andrew Cagney]

On Mon, 28 Sept 2020 at 22:49, Andrew Cagney <andrew.cagney at gmail.com> wrote:
>
>
>
> On Mon, 28 Sep 2020 at 21:53, Paul Wouters <paul at nohats.ca> wrote:
>>
>> On Mon, 28 Sep 2020, Andrew Cagney wrote:
>>
>> > I'm planning on removing the sanitizer ipsec-auto-up.n.sed.
>>
>> I understand why, but I also understand's Antony's point.
>>
>> To make it harder, once I rewrite the tests to also be documentation,
>> we have an additionally issue of adding weird things to the configs
>> people might copy. Like slow retransmit timeouts or impairs.
>
>
> slow-retransmits should be deleted (I thought I had, I see I hadn't)
> Your suspicion that IKEv1 didn't suppress retransmits turned out to be because the tests in question weren't trying to supress retransmits.

I've deleted the slow-retransmits hack.

>>
>>
>> I'm also a bit nervous if 90% of our tests use --impair
>> delete-on-retransmit At one point are we still testing "regular operation"
>> code with our tests.
>
>
> Two types of tests:
> 1. regression tests - these should be deterministic - someone reports a bug or we add a feature and we add a test for it
> 2. stress testing; the whole point of these is to be non-deterministic
> but we're discussing deterministic tests here
>
>>
>> Antony's filter in ipsec-auto-up.n.sed does address most of these
>> issues.
>>
>>
>
> This assertion is not true.  Please look at the examples I posted.  It silently removes important contextual information from test output - namely that a retransmit occurred - while leaving behind the consequences.  The upshot is that a retransmit still causes the test to fail; only that the retransmit happened is obscured.
>
> With the sanitizer disabled I got roughly the same number of fails.

I've added the sanitizer sanitize-retransmits.sed, with a line like:

   ipsec auto --up westnet-eastnet-ikev2 # sanitize-retransmits

it will strip retransmits from the output.  This way a test wanting
retransmits stripped from the output can do so explicitly.

(I've not removed ipsec-auto-up.n.sed)


>>
>> But other issues still remain, such as retransmits causing
>> different traffic counters in the output and causing false positives.
>>
>>
>> So, I guess I have no good answers or strong opinions here. Just
>> concerns :)
>>
>> Paul