[Swan-dev] pointless test failures

[Paul Wouters]

On Sun, 19 Jul 2015, D. Hugh Redelmeier wrote:

> netkey-audit-01:
>
> In west.console.diff, I see:
>
> +003 "ikev1-aggr": IKEv1 Aggressive Mode with PSK is vulnerable to dictionary attacks and is cracked on large scale by TLA's
>
> I imagine that this is expected.  Whoever changed this should fix the
> reference output.

Yes, I wasn't sure yet if we would leave it in like that but I guess we
should, and update the reference output. I have also been struck by the
output gathering bug that stopped showing me the test output graphics.

> More troubling: a lot of messages like
> -type=UNKNOWN[2408] msg=audit(XXX): pid=PID uid=0 auid=AUID ses=SES subj=system_u:system_r:init_t:s0 msg='op=....
> seem to have disappeared from east and west.
> Is this expected?

If you do not have a Makefile.inc.local with USE_LINUX_AUDIT=true then
yes.

> ================
>
> ikev2-01-fallback-ikev1:
>
> east:
> +000 "westnet-eastnet-ikev2-fallback":   retransmit-interval: 9999ms; retransmit-timeout: 99s;
>
> I imagine that this is expected.  Whoever changed this should fix the
> reference output.

We are still in a little bit if a discussion how to handle these. One
proposal is to add this to conn %default in every test case, the
other proposal was writing a sanitizer. Both have their pros and cons.

>
> ikev2-05-basic-psk:
>
> west:
> Someone added an ipsec status command.
>
> Whoever changed this should fix the reference output.

That was a commit accident. fixed.

> ================
>
> ikev2-delete-01 & ikev2-delete-01:
>
> west:
>
> a ref and refhim change.  We don't know what this is about.  I assume
> that it is a false positive.  We don't know how to eliminate this.
>
> It has been suggested that this was due to a change I made.  I have no
> knowledge of this.

refhim= went back to 0 which is good. and should be updated.

> ================
>
> ikev2-12-x509-ikev1:
>
> west:
> -000 "westnet-eastnet-ikev2" #1: starting keying attempt 2 of an unlimited number, but releasing whack
> +002 "westnet-eastnet-ikev2" #1: deleting state #1 (STATE_MAIN_I1)
>
> This looks like an intentional change.  Whoever changed this should
> fix the reference output.

looks like addition of impair--noretransmits

> ================
>
> At this point, it seems unrewarding for me to look at test results.
> (Except for sore thumbs like core files.)
>
> Please, folks, clean up the reference logs!  Useless failures are a big
> drain on all of us.

I agree. I have been discourages by running testruns for 7h and then
getting a python backtrace.....

Paul