[Swan-dev] f28 and testing's(f22) abysmal results

[Andrew Cagney]

A status update ...

On Tue, 25 Sep 2018 at 10:25, Andrew Cagney <andrew.cagney at gmail.com> wrote:
> So what can be done?  Several changes to the framework (I assume we
> don't want to disable electric fence) are:
>
> - on the theory that the HOST's KVM is too old, upgrade testing to
> something more recent, I thing that's been looked at

The machine was upgraded to F28.  That didn't go well:
- the build machine timed out during boot
(so re-build the VMs)
- swanfedora28base refuses to install
ttps://bugzilla.redhat.com/show_bug.cgi?id=1633267
I've tried reinstalling (and deleting) everything I could think of, but no luck.
Best guess is that there is still some hangover from f22, but what.

> - strip stuff from the boot (I should grep the test logs to confirm
> that this is where the slow down is)
>
> - (speculation) reduce boot verbosity - the test runner has to wade
> through all the boot messages
>
> And several changes to the tests:
>
> - add something like --impair delete-on-fail
> So that in tests where the responder is expected to reply with an
> error notification, the test can quickly be aborted.  Currently the
> tests are allowed to timeout.
>
> - continue to replace sleeps and 4-pings with things like one-ping or
> wait-until-alive and wait-for-whack-trafficstatus
> Unfortunately this is slow and tedious
>
> - revisit how timeouts are handled in the testsuite
> Up until now I've been using either a large timeout or --impair
> suppress-retransmits, to stop the re-transmits where that isn't
> relevant to the test.  With 160 more tests now having problems, an
> alternative strategy might be needed:
> - sanitize the standard re-transmit messages, and then for tests where
> the re-transmits matter 'impair' the message so it isn't sanitized
> - sanitize the retransmit impaired message so updating is easier
>
> Andrew