[Swan-dev] testing noise

[Paul Wouters]

On Thu, 18 Jun 2015, D. Hugh Redelmeier wrote:

> I found the output of this command interesting:
> 	grep -h '^[+-]' */OUTPUT/*.diff | sort | uniq -c | sort -nr | less
> It finds the changed lines, counts the duplication, and displays them
> in order of descending frequency.
>
> The top hits were:
>   1384 +src 0.0.0.0/0 dst 0.0.0.0/0
>    960 +road #
>    706 +       socket out priority 0 ptype main
>    706 +       socket in priority 0 ptype main
>    378 +west #
>    290 +east #
>    287 +rtt min/avg/max/mdev = 0.XXX/0.XXX/0.XXX/0.XXX ms
>    262 +--- 192.1.2.23 ping statistics ---
>    259 +2 packets transmitted, 2 received, 0% packet loss, time XXXX
>    248 +000

Those and the - minus below could be because of ondemand and
dpd/liveness. It's impossible to tell without context. I have been
thinking of and in 1 test case testing using a "single ping" redirected
to dev/null for these to reduce the variableness of the results

>    130 +000 nhelpers=-1, uniqueids=yes, perpeerlog=no, shunt_lifetime=900

This is a new option that just got merged in and needs to be added to
all test cases. I will do that today.

Paul

>     38 -4 packets transmitted, 4 received, 0% packet loss, time XXXX
>     38 +4 packets transmitted, 0 received, 100% packet loss, time XXXX
> These are the first, and they are way down on the list at position 82.

Again, context is needed to make any statement on these unfortunately,
which is what makes them hard.

Paul