[Swan-dev] test run's log prints EXPECTATION FAILED lines from another run

[D. Hugh Redelmeier]

| From: Antony Antony <antony at phenome.org>

| I will share my suspicion.

Thanks!

|  check your pluto version?  does it match with test run?

	[build at bluebird bluebird.mimosa.com]$ pwd
	/home/build/results/bluebird.mimosa.com
	[build at bluebird bluebird.mimosa.com]$ ls
	2015-05-24-bluebird.mimosa.com-v3.13oe6-190-ga8c95e1-dirty-oe-scratch3
	2015-05-25-bluebird.mimosa.com-v3.13oe6-190-ga8c95e1-dirty-oe-scratch3
	2015-05-26-bluebird.mimosa.com-v3.13oe6-210-ga84a741-dirty-oe-scratch3
	2015-06-03-bluebird.mimosa.com-v3.13aq6-346-g147d1f0-dirty-master
	2015-06-03-bluebird.mimosa.com-v3.13aq6-365-g0ed8082-dirty-master
	2015-06-04-bluebird.mimosa.com-v3.13aq6-365-g0ed8082-dirty-master
	2015-06-05-bluebird.mimosa.com-v3.14rc1-16-gcc40c56-dirty-master

The make's logfile said:
	results will be in /home/build/results/bluebird.mimosa.com/2015-06-05-bluebird.mimosa.com-v3.14rc1-16-gcc40c56-dirty-master
This was line 590 of 36924 of the log.  So I don't think that the
versions were confused.

I looked at the versions on the VMs, but that doesn't count because
between the full run and now, I changed pluto a bit and updated the
machines.  I then ran one test.

Within my source directory, when I do
	ls -ltr testing/pluto/*/OUTPUT/*.pluto.log
I see only 14 lines.
===> Why would that be?
I depend on those files!

Within
	~/results/bluebird.mimosa.com/2015-06-05-bluebird.mimosa.com-v3.14rc1-16-gcc40c56-dirty-master
when I do
	ls -ltr */OUTPUT/*.pluto.log
I see 512 lines.  And within those files, there are 1781 lines
matching EXPECTATION FAILED (1758 due to my changes!).  They boil down
to 3 distinct pexpects failing and ALL of them refer to the correct
lines in the source.

| If it does not match install failed. That is my guess.

I think my observations (above) suggest that the installation
succeeded but that the scripting is confused.  I must admit that I'm not 
looking at the reporting, I'm only looking at the greps done in the 
transcripts, so maybe the reporting is correct.

Is there anything else that you'd like me to check?