[Swan-dev] test script limitation

[Andrew Cagney]

(I'm referring to the *.sh scripts within a test)

Traditional test directories, such as dpd-02, contain the script files:

$ ls *.sh
eastinit.sh
final.sh
westinit.sh
westrun.sh

so that more complicated sequences could be built, this was extended to
allow arbitrary names with a lexicographic ordering.  This way, the above
can be broken down and carefully sequenced making it more robust:

$ ls -1 *.sh
01-east-init.sh
02-west-init.sh
03-west-up.sh
04-east-established.sh
05-west-ping.sh
06-west-block.sh
07-east-down.sh
08-west-unblock.sh
09-west-ping.sh
final.sh

(It's an IKEv1 test so the test needs to check that east is established
before west can send a ping, the file names I'm using are arbitrary).

While it works, I'm finding it very much write-only: the *.console.txt
files don't reflect the above order making deciphering failures confusing
and updates fragile.  For instance, if there's a problem on west, the first
difference is on east (because it is compared first).

I'm now wondering if having a merged file (both input and output) so
everything can be read inline would be better:

east# ipsec start
[initsystem stuff]
east# wait-for-pluto
east#
west# ipsec start
[initsystem stuff]
west# wait-for-pluto
west# ping-once --up east
up
west#

(we want to make writing tests easy).  In this scenario, the differences
could be shown in context.

just playing with ideas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libreswan.org/pipermail/swan-dev/attachments/20211013/eba05b4e/attachment.html>