[Swan-dev] testing: drifting reference logs

[Antony Antony]

FYI: interop-ikev2-racoon-03-psk-initiator  
break "make check", because it has an unexpected file eastrun.sh 
It came from a recenct umlpluto to kvmconversion, ffc7bc4d
I am disabling it in TESTLIST; careful while converting old tests!

-antony


On Fri, Jan 02, 2015 at 02:36:12PM -0500, Paul Wouters wrote:
> On Fri, 2 Jan 2015, Antony Antony wrote:
> 
> >We have another output change that affect almost all test cases(200+) now.
> >
> >-000 algorithm IKE encrypt: v1id=13, v1name=DISABLED-OAKLEY_AES_CTR, v2id=13, v2name=AES_CTR, blocksize=16, keydeflen=128
> >+000 algorithm IKE encrypt: v1id=8, v1name=OAKLEY_CAMELLIA_CBC, v2id=23, v2name=CAMELLIA_CBC, blocksize=16, keydeflen=128
> 
> It should be more than just these two lines, as both OAKLEY_AES_CTR and
> OAKLEY_CAMELLIA_CBC got enabled recently.
> 
> >I suggest output update only if the test pass.  That would increase the quality.
> 
> I'm working through a full test run still after I merged in the CTR code
> from Andrew.
> 
> >My feeling about WIP test cases is do not commit output.
> 
> I guess we all still disagree on the best method...... I've changed the
> output but will do a full testrun before commiting the updated output.
> 
> >Paul if you are going to convert all kvm test cases, first make output zero byte file. And only after they pass, preferably on more than one machine, commit the output. Otherwise, from the past 6 months of experience, it feels like a lot of work to maintain them.
> 
> I just removed all console output files from tests marked as umltest
> (some exceptions for some ipv6 tests which i know were converted but
> was not marked as kvmplutotest)
> 
> If the full test run goes well, tonight the tests should be passing
> again. Sorry for the noise,
> 
> Paul
>