[Swan-dev] make clean is eating my testing/pluto/*/OUTPUT directories

[D. Hugh Redelmeier]

| From: Andrew Cagney <andrew.cagney at gmail.com>

I approach this as a grumpy, superstitious, and unreasonable user.  I'm 
very glad that you do not.

| (I suspect your output is in /home/build/..... somewhere).

Maybe.  That too is a mechanism with magic I don't understand.  For
example, if I run a single test, does its output get squirelled away?
If I run the same test twice in a day, does the second overwrite the
first?  I think things are moderately odd when the clock strikes
midnight during a multiple-test run.  Even though I'm superstitious, I
don't understand what's special about midnight.

| so the intent (since 2005) is been to remove OUTPUT during "clean".
| However, looking more carefully at the above output, we see the
| recursive "make clean" was run in OBJDIR, so didn't actually do very
| much :-/

That may be a bug, but to me it was a feature.

| While I can see about changing recursive "make clean" to not delete
| OUTPUT, I suspect having "make check" delete it anyway isn't the
| behaviour you're looking for?

As a superstitious user, I run spells.  The only spell I know and use
with make check is
	date ; make check UPDATE=1 ; date
Darned if I know what the UPDATE=1 means, but I do know I need it. 
Notationally, I think that it is dumb.

| So back to my question, when should ${srcdir}/**/OUTPUT be deleted?
| 
| 
| First, it is probably worth noting some of swantest's, er,  features:
| 
| - by default, it stores results in /home/build/...
| try to think of OUTPUT as a scratch directory; not working? try harder -)

OUTPUT is a scratch directory in some sense.  We certainly don't want
it in the git repository.  But it is in the source tree.  Just like
OBJ.*.

I'm used to */OUTPUT persisting until I do something with intent to
change it.

I'm resisting /home/build/... See excuses above and in other whiny
messages.

| - by default, it is incremental and only runs a test if
| /home/build/... suggests that the test failed or wasn't run
| this means "make check" "make check", even after deleting OUTPUT,
| re-runs just the tests that fail; that is useful
| I'm guessing that, by deleting OUTPUT, it is easier to see swantest's progress?
| On the other hand, deleting OUTPUT not triggering a test re-run is
| really confusing (so confusing that I hacked swantest to spit out the
| path to the directory I needed to delete); yes, I know, I need to try
| harder at thinking of OUTPUT as a scratch directory

No strong opinion.  Mostly because this usage isn't within my
collection of spells.

I don't trust the mechanisms because they so often have surprised me.
Certainly the scripts' idea of failure seems very unreliable.  Basing
file deletion or other serious action on that is pretty scary.

| - it uses a heuristic based on git to determine if something changed
| and the tests should be re-run from scratch (and a new directory in
| /home/build/... should be created)
| The heuristic isn't very robust.

The heuristics are inviting unpleasant surprises.  At least that's my
guess.

| - not deleting OUTPUT (least surprise)
| Unfortunately this makes it harder to see how far the teastsuite has
| progressed; perhaps just delete the directories that failed ...

Timestamps are available.  pluto-testlist-scan.sh (my hacked together
spell) notes when time goes backwards between two tests in the fixed
order of TESTLIST.  That works for me as a reader of the report.

| - a slightly more robust check to detect changes and determine that
| all tests need to be re-run
| This proved less useful than you might expect; I've found that I
| instead want incremental test runs, regardless of what changed, as the
| default - it lets me incrementally fix and re-run without re-running
| everything

Yes, for sure

| so, what's your preference?

Less magic / surprise / danger.  I'm in favour of simplicity and
understandability.

Capture idiomatic use that seems valuable.

Don't spontaneously loose work.

Bonus: document and aid the spread of effective methods people
actually use.  This might allow followers to get productive more
quickly (or at all).  Of course some of us only read documentation
sporadically.

We've gone from the test suite being non-functional to it being
valuable within our own group.  The wiki instructions mostly work for
me (an ignorant user, but with access to Paul).  It would be great to
make it workable for people who don't know Paul or you or Antony.


One of my spells has been to run make clean before each make programs
(oops, make base).  The dependencies (used to?) have enough bugs that
this prevented annoying surprises.

One of my related spells has been to
	rm OBJ<tab> -r
because it seems to be necessary sometimes, even though make clean
doesn't (always?) do it.  For example, I sometimes build on the host
and sometimes build on the guest machines, and they share OBJ* very
badly.  I think that makefile fragments or dependencies live there
that are wrong for the other environment, causing make to break.

(Aside: I've recently come to write -r in rm AFTER the pathnames since
it seems likely to reduce accidents.  See?  I can learn new tricks.)

Why would I make on the host when I have guests?  Because it is a fast
check for stupid mistakes in the code.  And the error reporting is
more convenient (using host pathnames, for example).

If I had a better model of the whole thing, changes would not throw me
as much.  As a superstitious user, I don't take well to my spells
breaking.