[Swan-dev] swantest bug

[D. Hugh Redelmeier]

I ran a "make UPDATE=1 check".  It failed with a python error about seven
hours later.  It is in the code that summarizes the results.

================
write text summary file /home/build/results/bluebird.mimosa.com/2015-07-19-bluebird.mimosa.com-v3.14rc3-49-g9fb59ee-dirty-master/table.txt
Traceback (most recent call last):
  File "../utils/swantest", line 2212, in <module>
    main()
  File "../utils/swantest", line 2197, in main
    gen_graphs_tables(args)
  File "../utils/swantest", line 2150, in gen_graphs_tables
    s.scan_test_runs()
  File "../utils/swantest", line 497, in scan_test_runs
    self.scan_test_results()
  File "../utils/swantest", line 684, in scan_test_results
    e = self.create_test_run_table()
  File "../utils/swantest", line 762, in create_test_run_table
    self.write_txt(table)
  File "../utils/swantest", line 782, in write_txt
    line = self.make_test_line(test)
  File "../utils/swantest", line 811, in make_test_line
    test["result"], test["expect"], test["name"], st1, st2)
KeyError: 'expect'
Makefile:33: recipe for target 'check' failed
make[2]: *** [check] Error 1
make[2]: Leaving directory '/home/build/libreswan/testing/pluto'
../mk/subdirs.mk:33: recipe for target 'check' failed
================

Line 811 in swantest is obviously wrong: it is in an ELSE clause
of an IF that tests for "expect" in test.  So we know there is no
"expect", and yet 811 tries to print test["expect"].

This chunk of swantest has not changed in half a year.  It has always
been wrong.  Why is it hitting me now?

There is a hint in commit 58d2fa75746b83954ae7865f221acc3bd66789ba.
That included this hunk:

================
@@ -688,7 +700,11 @@ class scantests:
         for key, test in self.tests.items():
             row = []
             row.append(test["name"])
-            row.append(test["expect"])
+            if "expect" in test:
+                row.append(test["expect"])
+            elif "output" in test:
+                row.append(test["output"])
+                               
             if "result" in test:
                 row.append(test["result"])
             else:
================

It sure looks as if test["expect"] is no longer unconditionally used.
(The last added line has trailing whitespace. "git show" shows it in red.)

What is the correct fix to swantest?

After I fix this, how should I resume the tests summary process
without starting the tests all over again?