[Swan-dev] testing: drifting reference logs

[Andrew Cagney]

Perhaps the attached sanitizer will help.  It changes this:

 000
 000 IKE algorithms supported:
-000
-000 algorithm IKE encrypt: v1id=5, v1name=OAKLEY_3DES_CBC, v2id=3,
v2name=3DES, blocksize=8, keydeflen=192
-000 algorithm IKE encrypt: v1id=24, v1name=OAKLEY_CAMELLIA_CTR,
v2id=24, v2name=CAMELLIA_CTR, blocksize=16, keydeflen=128
-000 algorithm IKE encrypt: v1id=8, v1name=OAKLEY_CAMELLIA_CBC,
v2id=23, v2name=CAMELLIA_CBC, blocksize=16, keydeflen=128
-000 algorithm IKE encrypt: v1id=13, v1name=OAKLEY_AES_CTR, v2id=13,
v2name=AES_CTR, blocksize=16, keydeflen=128
-000 algorithm IKE encrypt: v1id=7, v1name=OAKLEY_AES_CBC, v2id=12,
v2name=AES_CBC, blocksize=16, keydeflen=128
-000 algorithm IKE encrypt: v1id=65004, v1name=OAKLEY_SERPENT_CBC,
v2id=65004, v2name=SERPENT_CBC, blocksize=16, keydeflen=128
-000 algorithm IKE encrypt: v1id=65005, v1name=OAKLEY_TWOFISH_CBC,
v2id=65005, v2name=TWOFISH_CBC, blocksize=16, keydeflen=128
-000 algorithm IKE encrypt: v1id=65289, v1name=OAKLEY_TWOFISH_CBC_SSH,
v2id=65289, v2name=TWOFISH_CBC_SSH, blocksize=16, keydeflen=128
-000 algorithm IKE hash: id=1, name=OAKLEY_MD5, hashlen=16
-000 algorithm IKE hash: id=2, name=OAKLEY_SHA1, hashlen=20
-000 algorithm IKE hash: id=4, name=OAKLEY_SHA2_256, hashlen=32
-000 algorithm IKE hash: id=5, name=OAKLEY_SHA2_384, hashlen=48
-000 algorithm IKE hash: id=6, name=OAKLEY_SHA2_512, hashlen=64
-000 algorithm IKE hash: id=9, name=DISABLED-OAKLEY_AES_XCBC, hashlen=16
-000 algorithm IKE dh group: id=2, name=OAKLEY_GROUP_MODP1024, bits=1024
-000 algorithm IKE dh group: id=5, name=OAKLEY_GROUP_MODP1536, bits=1536
-000 algorithm IKE dh group: id=14, name=OAKLEY_GROUP_MODP2048, bits=2048
-000 algorithm IKE dh group: id=15, name=OAKLEY_GROUP_MODP3072, bits=3072
-000 algorithm IKE dh group: id=16, name=OAKLEY_GROUP_MODP4096, bits=4096
-000 algorithm IKE dh group: id=17, name=OAKLEY_GROUP_MODP6144, bits=6144
-000 algorithm IKE dh group: id=18, name=OAKLEY_GROUP_MODP8192, bits=8192
-000 algorithm IKE dh group: id=22, name=OAKLEY_GROUP_DH22, bits=1024
-000 algorithm IKE dh group: id=23, name=OAKLEY_GROUP_DH23, bits=2048
-000 algorithm IKE dh group: id=24, name=OAKLEY_GROUP_DH24, bits=2048
 000

to this:

 000
 000 IKE algorithms supported:
+000
+000 [...]
 000

while having a little barf if it finds that not all lines match.  I've
several patches looming that further change the above output, having
something like this added should make integrating them easier.

With a bit of care it can also be used to directly update the existing
*.console.txt files in the repo

Andrew

PS:   Perhaps [...] should be changed to [see algorithm-ike-sanitize.awk] :-)

On 2 January 2015 at 14:36, Paul Wouters <paul at nohats.ca> wrote:
> On Fri, 2 Jan 2015, Antony Antony wrote:
>
>> We have another output change that affect almost all test cases(200+) now.
>>
>> -000 algorithm IKE encrypt: v1id=13, v1name=DISABLED-OAKLEY_AES_CTR,
>> v2id=13, v2name=AES_CTR, blocksize=16, keydeflen=128
>> +000 algorithm IKE encrypt: v1id=8, v1name=OAKLEY_CAMELLIA_CBC, v2id=23,
>> v2name=CAMELLIA_CBC, blocksize=16, keydeflen=128
>
>
> It should be more than just these two lines, as both OAKLEY_AES_CTR and
> OAKLEY_CAMELLIA_CBC got enabled recently.
>
>> I suggest output update only if the test pass.  That would increase the
>> quality.
>
>
> I'm working through a full test run still after I merged in the CTR code
> from Andrew.
>
>> My feeling about WIP test cases is do not commit output.
>
>
> I guess we all still disagree on the best method...... I've changed the
> output but will do a full testrun before commiting the updated output.
>
>> Paul if you are going to convert all kvm test cases, first make output
>> zero byte file. And only after they pass, preferably on more than one
>> machine, commit the output. Otherwise, from the past 6 months of experience,
>> it feels like a lot of work to maintain them.
>
>
> I just removed all console output files from tests marked as umltest
> (some exceptions for some ipv6 tests which i know were converted but
> was not marked as kvmplutotest)
>
> If the full test run goes well, tonight the tests should be passing
> again. Sorry for the noise,
>
>
> Paul
> _______________________________________________
> Swan-dev mailing list
> Swan-dev at lists.libreswan.org
> https://lists.libreswan.org/mailman/listinfo/swan-dev
-------------- next part --------------
A non-text attachment was scrubbed...
Name: algorithm-ike-sanitize.awk
Type: application/x-awk
Size: 2987 bytes
Desc: not available
URL: <https://lists.libreswan.org/pipermail/swan-dev/attachments/20150106/482af21f/attachment.bin>