[Swan] Running KVM test cases, was Re: Who changed parser.l ????

[Wes Hardaker]

Paul Wouters <pwouters at redhat.com> writes:

>> 1) a minimal set that just does config file parsing and other, um,
>>   simple tests that doesn't require a complete tunnel setup.
>>   + can be run by anyone, no root access required
>
> Those are tests in testing/scripts/readwrite*

The problem is that those tests read a file, attempt to write stuff back
out but don't actually check that what is read in is what is expected to
have been read.  IE, when something goes wrong you wouldn't know it
unless it dies badly (as opposed to incorrectly reading the data).

Some of the tests do involve inserting bad data and making sure there is
an error (eg -25), but it still exits(0) and it took me a bit to realize
that it's checking for an error and the error is getting found in the
.conf file and printed, but what's checking that all the errors expected
to be caught actually are?

Yes, the TESTLIST has a list of "this one should pass, this one should
fail" but it isn't a complete bit-for-bit check, which I think would be
good to have.

(and mildly-difficult to write, so I understand why its not there, mind
you...  I'm just expounding for more testing.  I've come to believe,
over the years, that there is never 'too much testing').
-- 
Wes Hardaker                                     
My Pictures:  http://capturedonearth.com/
My Thoughts:  http://pontifications.hardakers.net/