[Swan-dev] testing, testing

[Andrew Cagney]

I suspect that the algorithm is something like:

for try in 1..5:
   for test in tests:
      archive = form-archive-directory-from test + date + git + ...
      if no archive or not archive passed
          delete OUTPUT
          run test
          copy OUTPUT to archive

So failing tests are always attempted multiple times.  It is just
that, when the time changes, the archive directory also changes
causing previously successful tests to also be re-attempted.

Passing:
  --retry 0
to swantest might help.

I'm beginning to think that the best default behaviour might be to
only attempt tests with no OUTPUT directory. It means:
- tests are only attempted once (so intermittent failures are not hidden)
- re-running a test is easy - delete OUTPUT
I'll try that locally for a while.

Andrew


On 15 June 2015 at 09:26, D. Hugh Redelmeier <hugh at mimosa.com> wrote:
> | From: D. Hugh Redelmeier <hugh at mimosa.com>
>
> | Why the heck are tests being run twice?  When will it stop?
>
> I ran the tests this morning.  Started at 1:29 local time; ended at 8:35.
> Each test that was run was run only once.
>
> I wonder what was different.
> _______________________________________________
> Swan-dev mailing list
> Swan-dev at lists.libreswan.org
> https://lists.libreswan.org/mailman/listinfo/swan-dev