From dee at poczta.onet.pl  Thu Aug  1 13:05:42 2019
From: dee at poczta.onet.pl (dee at poczta.onet.pl)
Date: Thu, 01 Aug 2019 15:05:42 +0200
Subject: [Swan-dev] Libreswan 3.29 compilation error
Message-ID: <121067528-f991542e80ca4f7bc4217504af8ee320@pmq1v.m5r2.onet>

Dear Colleagues,?
?
I would like to report a compilation error of Libreswan 3.29 on Slackware 14.2 64-bit kernel 4.4.186:
?
make[2]: Entering directory '/usr/local/src/libreswan-3.29/lib/libipsecconf'
[...]
-c /usr/local/src/libreswan-3.29/lib/libipsecconf/confread.c
/usr/local/src/libreswan-3.29/lib/libipsecconf/confread.c: In function 'ipsecconf_default_values':
/usr/local/src/libreswan-3.29/lib/libipsecconf/confread.c:68:54: error: expected expression before ')' token
?# define SOPT(kbf, v)? { cfg->setup.options[kbf] = (v) ; }
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ^
/usr/local/src/libreswan-3.29/lib/libipsecconf/confread.c:82:2: note: in expansion of macro 'SOPT'
? SOPT(KBF_XFRMLIFETIME, XFRM_LIFETIME_DEFAULT); /* not used by pluto itself */
? ^
../../mk/depend.mk:34: recipe for target 'confread.o' failed
make[3]: *** [confread.o] Error 1
../../mk/targets.mk:82: recipe for target 'all' failed
make[2]: *** [all] Error 2
?
Directives used?USE_GLIBC_KERN_FLIP_HEADERS?=true in config.mk, trying to build with KLIPS.?
I appreciate your support.
?
Regards
?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libreswan.org/pipermail/swan-dev/attachments/20190801/1b184083/attachment.html>

From andrew.cagney at gmail.com  Sat Aug  3 13:55:56 2019
From: andrew.cagney at gmail.com (Andrew Cagney)
Date: Sat, 3 Aug 2019 09:55:56 -0400
Subject: [Swan-dev] proposed patch to re-create version.c every time
In-Reply-To: <20190731125512.jc4m3eqrelokdlla@AntonyAntony.local>
References: <20190731125512.jc4m3eqrelokdlla@AntonyAntony.local>
Message-ID: <CAJeAr6tKPgt7wKcMuxb7jmMc8Zhoe1SQKavvwK0H4ytEPb_f4Q@mail.gmail.com>

On Wed, 31 Jul 2019 at 08:56, Antony Antony <antony at phenome.org> wrote:

> Hi
> tangential to "make showversion" discussion, there is an issue that annoys
> me. Now I have a fix for it! If there is no violent opposition against
> this
> proposed patch I would like to apply the attached patch.
>
> version.c dependency is broken while developing and testing continuously
> from
> one directory. Current dependency works great if you always build from a
> clean git tree or run 'make clean' first or build from a tar ball.
> However,
> less ideal for continuous development and testing.
>
>
Right, for the case where the version file really matters, it is correct.
 See the bottom end of:
https://lists.libreswan.org/pipermail/swan-dev/2016-August/001609.html
This is also the least worst option.

For instance, invoking 'make' on an up-to-date build should do 'nothing'.
With this patch applied, repeated makes will instead repeatedly rebuild all
the programs.  In addition to being confusing, and misleading (what did I
change, is a dependency broken) it is also wrong - it cascades as now the
re-built programs are going to needlessly trigger other dependencies (for
instance make check in cavp).

On KVMs, things get worse.  A sequence like (i.e., make certain things are
up-to-date and then run a test):
   $ make kvm-install kvm-test KVM_TESTS=testing/pluto/basic-pluto-01
now has the additional overhead of both running git with the dirty check
and against 9p (which I think we can all agree is expensive, but something
we can't see since it is being hidden under a rock) and then build new
executable.

The problem with git and KVM is also also why I've never bothered to fix
this - any 'technically correct' solution will need to invoke 'git dirty'
within the KVM - is that overhead really worth it - not for me?

Andrew

Here is the case where it breaks.
> You run "make base install-base" starting with a clean git tree.
> Now make changes to the code and commit those changes.
> then "make base install-base". Now the  version.c would be stale.
> ie. pluto --version would not change even though the code changed, and
> commit id, or "make showversion" changed.
>
> ./OBJ.linux.x86_64/programs/pluto/pluto --version
> Libreswan v3.28-526-g5c9063def5-master-s2 XFRM(netkey) esp-hw-offload FORK
> PTHREAD_SETSCHEDPRIO NSS (IPsec profile) DNSSEC FIPS_CHECK LABELED_IPSEC
> SECCOMP LIBCAP_NG LINUX_AUDIT XAUTH_PAM NETWORKMANAGER CURL(non-NSS)
>
> With this proposed patch version.c will be always (re)created.
> "make base' or make programs will have the correct version string.
> It would help me a lot. Are there any side effects if version.c is
> (re)created every time?
>
> Currently I use a work around, "make clean". it has two disadvantages.
> It would add another 30 - 180 sec delay for when you make a change
> re-compile.  Often I forget that and have back track double check.
>
> Let me know what you think. If there are no major objections to this patch
> I
> will apply it, if no comment in week I will apply it:)
> See the attached patch.
>
> -antony
> _______________________________________________
> Swan-dev mailing list
> Swan-dev at lists.libreswan.org
> https://lists.libreswan.org/mailman/listinfo/swan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libreswan.org/pipermail/swan-dev/attachments/20190803/c6313b1c/attachment.html>

From andrew.cagney at gmail.com  Mon Aug  5 17:33:39 2019
From: andrew.cagney at gmail.com (Andrew Cagney)
Date: Mon, 5 Aug 2019 13:33:39 -0400
Subject: [Swan-dev] savage regression in test results
Message-ID: <CAJeAr6tGz5TeEb7EFks1eDsym=wRMQuwd_Cq0Gtaz=FsC7z3-w@mail.gmail.com>

Channelling Hugh ....

The results were down to ~80 but then, with recent changes, went into the
toilet.  While they've clawed their way back a bit they are still hovering
around ~130
https://testing.libreswan.org/

Andrew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libreswan.org/pipermail/swan-dev/attachments/20190805/4f582323/attachment.html>

From paul at nohats.ca  Mon Aug  5 18:26:19 2019
From: paul at nohats.ca (Paul Wouters)
Date: Mon, 5 Aug 2019 14:26:19 -0400
Subject: [Swan-dev] savage regression in test results
In-Reply-To: <CAJeAr6tGz5TeEb7EFks1eDsym=wRMQuwd_Cq0Gtaz=FsC7z3-w@mail.gmail.com>
References: <CAJeAr6tGz5TeEb7EFks1eDsym=wRMQuwd_Cq0Gtaz=FsC7z3-w@mail.gmail.com>
Message-ID: <8B5F7BD5-7143-41A3-8627-FC7D5B303B3E@nohats.ca>

I am aware - the good thing is we are on f30, ready for rhel8/centos8 and can pick namespaces or kvm with roughly the same result. I?m working on getting us back to ?normal?

Sent from mobile device

> On Aug 5, 2019, at 13:33, Andrew Cagney <andrew.cagney at gmail.com> wrote:
> 
> Channelling Hugh ....
> 
> The results were down to ~80 but then, with recent changes, went into the toilet.  While they've clawed their way back a bit they are still hovering around ~130
> https://testing.libreswan.org/
> 
> Andrew
> 
> _______________________________________________
> Swan-dev mailing list
> Swan-dev at lists.libreswan.org
> https://lists.libreswan.org/mailman/listinfo/swan-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libreswan.org/pipermail/swan-dev/attachments/20190805/5649f332/attachment-0001.html>

From andrew.cagney at gmail.com  Mon Aug  5 20:16:31 2019
From: andrew.cagney at gmail.com (Andrew Cagney)
Date: Mon, 5 Aug 2019 16:16:31 -0400
Subject: [Swan-dev] savage regression in test results
In-Reply-To: <8B5F7BD5-7143-41A3-8627-FC7D5B303B3E@nohats.ca>
References: <CAJeAr6tGz5TeEb7EFks1eDsym=wRMQuwd_Cq0Gtaz=FsC7z3-w@mail.gmail.com>
 <8B5F7BD5-7143-41A3-8627-FC7D5B303B3E@nohats.ca>
Message-ID: <CAJeAr6sZbd2guDMx6PFMXs5PRoHaGOumkUcOsnzOTUgYStzVNw@mail.gmail.com>

On Mon, 5 Aug 2019 at 14:26, Paul Wouters <paul at nohats.ca> wrote:

> I am aware - the good thing is we are on f30, ready for rhel8/centos8 and
> can pick namespaces or kvm with roughly the same result. I?m working on
> getting us back to ?normal?
>
>
KVM_OS points at fedora28?

Sent from mobile device
>
> On Aug 5, 2019, at 13:33, Andrew Cagney <andrew.cagney at gmail.com> wrote:
>
> Channelling Hugh ....
>
> The results were down to ~80 but then, with recent changes, went into the
> toilet.  While they've clawed their way back a bit they are still hovering
> around ~130
> https://testing.libreswan.org/
>
> Andrew
>
> _______________________________________________
> Swan-dev mailing list
> Swan-dev at lists.libreswan.org
> https://lists.libreswan.org/mailman/listinfo/swan-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libreswan.org/pipermail/swan-dev/attachments/20190805/db7a5251/attachment.html>

From hugh at mimosa.com  Mon Aug  5 22:04:19 2019
From: hugh at mimosa.com (D. Hugh Redelmeier)
Date: Mon, 5 Aug 2019 18:04:19 -0400 (EDT)
Subject: [Swan-dev] pluto: Change some connection loading IKE version
	checks
Message-ID: <alpine.LFD.2.21.1908051801010.3183@redeye.mimosa.com>

I don't understand why 211c2b7b1fce9c72f15e68de6b69580c050e954d is a good 
idea.

The old tests were of the form "if we are not the right version, complain"
The new versions are of the form "if we are a particular wrong version, 
complain"

The original versions seem more robust.

Paul: can you explain why you think that the new version is an 
improvement?

From paul at nohats.ca  Mon Aug  5 22:51:30 2019
From: paul at nohats.ca (Paul Wouters)
Date: Mon, 5 Aug 2019 18:51:30 -0400 (EDT)
Subject: [Swan-dev] pluto: Change some connection loading IKE version
 checks
In-Reply-To: <alpine.LFD.2.21.1908051801010.3183@redeye.mimosa.com>
References: <alpine.LFD.2.21.1908051801010.3183@redeye.mimosa.com>
Message-ID: <alpine.LRH.2.21.1908051847470.2236@bofh.nohats.ca>

On Mon, 5 Aug 2019, D. Hugh Redelmeier wrote:

> I don't understand why 211c2b7b1fce9c72f15e68de6b69580c050e954d is a good
> idea.
>
> The old tests were of the form "if we are not the right version, complain"
> The new versions are of the form "if we are a particular wrong version,
> complain"
>
> The original versions seem more robust.
>
> Paul: can you explain why you think that the new version is an
> improvement?

Most of the tests are for features we only support in IKEv2, which
really means "from IKEv2 onwards". For example, we would expect
IKEv2.1 or IKEv3 to still support MOBIKE or RFC7427 style Digitial
Signatures.

Once we would implement IKEv2.1 or IKEv3, we would only need to look
at the difference between IKEv2 and IKEv3. The old code would however
break everything because it wrongly assumes "not IKEv2" means "IKEv1".

Paul

From paul at nohats.ca  Mon Aug  5 22:56:54 2019
From: paul at nohats.ca (Paul Wouters)
Date: Mon, 5 Aug 2019 18:56:54 -0400 (EDT)
Subject: [Swan-dev] savage regression in test results
In-Reply-To: <CAJeAr6sZbd2guDMx6PFMXs5PRoHaGOumkUcOsnzOTUgYStzVNw@mail.gmail.com>
References: <CAJeAr6tGz5TeEb7EFks1eDsym=wRMQuwd_Cq0Gtaz=FsC7z3-w@mail.gmail.com>
 <8B5F7BD5-7143-41A3-8627-FC7D5B303B3E@nohats.ca>
 <CAJeAr6sZbd2guDMx6PFMXs5PRoHaGOumkUcOsnzOTUgYStzVNw@mail.gmail.com>
Message-ID: <alpine.LRH.2.21.1908051852380.2236@bofh.nohats.ca>

On Mon, 5 Aug 2019, Andrew Cagney wrote:

> On Mon, 5 Aug 2019 at 14:26, Paul Wouters <paul at nohats.ca> wrote:
>       I am aware - the good thing is we are on f30, ready for rhel8/centos8 and can pick namespaces or kvm with roughly the same
>       result. I?m working on getting us back to ?normal?
> 
> 
> KVM_OS points at fedora28?

Wasn't that the OS of the host? not of the guests?
I think I changed libreswan-web/slave/Makefile.inc.local to KVM_OS=fedora30

I've been confused with the KVM_OS / KVM_VARIANT_OS variables before
especially in the testing.libreswan master/slave setup.

Paul

From tis at foobar.fi  Tue Aug  6 11:28:07 2019
From: tis at foobar.fi (Tuomo Soini)
Date: Tue, 6 Aug 2019 14:28:07 +0300
Subject: [Swan-dev] savage regression in test results
In-Reply-To: <alpine.LRH.2.21.1908051852380.2236@bofh.nohats.ca>
References: <CAJeAr6tGz5TeEb7EFks1eDsym=wRMQuwd_Cq0Gtaz=FsC7z3-w@mail.gmail.com>
 <8B5F7BD5-7143-41A3-8627-FC7D5B303B3E@nohats.ca>
 <CAJeAr6sZbd2guDMx6PFMXs5PRoHaGOumkUcOsnzOTUgYStzVNw@mail.gmail.com>
 <alpine.LRH.2.21.1908051852380.2236@bofh.nohats.ca>
Message-ID: <20190806142807.39951664@tuomo.foobar.fi>

On Mon, 5 Aug 2019 18:56:54 -0400 (EDT)
Paul Wouters <paul at nohats.ca> wrote:

> On Mon, 5 Aug 2019, Andrew Cagney wrote:
> 
> > On Mon, 5 Aug 2019 at 14:26, Paul Wouters <paul at nohats.ca> wrote:
> >       I am aware - the good thing is we are on f30, ready for
> > rhel8/centos8 and can pick namespaces or kvm with roughly the same
> > result. I?m working on getting us back to ?normal?
> > 
> > 
> > KVM_OS points at fedora28?  
> 
> Wasn't that the OS of the host? not of the guests?
> I think I changed libreswan-web/slave/Makefile.inc.local to
> KVM_OS=fedora30
> 
> I've been confused with the KVM_OS / KVM_VARIANT_OS variables before
> especially in the testing.libreswan master/slave setup.

Correct naming would be KVM_HOST_OS and KVM_GUEST_OS.


-- 
Tuomo Soini <tis at foobar.fi>
Foobar Linux services
+358 40 5240030
Foobar Oy <https://foobar.fi/>

From andrew.cagney at gmail.com  Tue Aug  6 12:45:39 2019
From: andrew.cagney at gmail.com (Andrew Cagney)
Date: Tue, 6 Aug 2019 08:45:39 -0400
Subject: [Swan-dev] savage regression in test results
In-Reply-To: <20190806142807.39951664@tuomo.foobar.fi>
References: <CAJeAr6tGz5TeEb7EFks1eDsym=wRMQuwd_Cq0Gtaz=FsC7z3-w@mail.gmail.com>
 <8B5F7BD5-7143-41A3-8627-FC7D5B303B3E@nohats.ca>
 <CAJeAr6sZbd2guDMx6PFMXs5PRoHaGOumkUcOsnzOTUgYStzVNw@mail.gmail.com>
 <alpine.LRH.2.21.1908051852380.2236@bofh.nohats.ca>
 <20190806142807.39951664@tuomo.foobar.fi>
Message-ID: <CAJeAr6vxv4DjsxM0iE1kHza+UPOZi8+MgFm5ADoYiibV+ff3Lw@mail.gmail.com>

On Tue, 6 Aug 2019 at 07:34, Tuomo Soini <tis at foobar.fi> wrote:

> On Mon, 5 Aug 2019 18:56:54 -0400 (EDT)
> Paul Wouters <paul at nohats.ca> wrote:
>
> > On Mon, 5 Aug 2019, Andrew Cagney wrote:
> >
> > > On Mon, 5 Aug 2019 at 14:26, Paul Wouters <paul at nohats.ca> wrote:
> > >       I am aware - the good thing is we are on f30, ready for
> > > rhel8/centos8 and can pick namespaces or kvm with roughly the same
> > > result. I?m working on getting us back to ?normal?
> > >
> > >
> > > KVM_OS points at fedora28?
>
>
as in:

   mk/kvm-targets.mk:KVM_OS = fedora28


> > Wasn't that the OS of the host? not of the guests?
> > I think I changed libreswan-web/slave/Makefile.inc.local to
> > KVM_OS=fedora30
>

It's the guest (the test framework should be host agnostic).

This explains why trying to compare local results with testing has been an
exercise in frustration - it's not been testing top-of-tree.

Per Tuomo's suggestion, I'll rename KVM_OS to KVM_GUEST_OS (and
KVM_VARIANT_OS which is a hack to pacify a libvirt warning).

We've an f30 tester set up under f30/ (see earlier posts).  I'll re-start
it as in:

  cd f30 ; rm -f nohup.out ; nohup master/testing/web/tester.sh slave/
results/ &

I've also made results/ a soft-link to ../testing/f30 so that they can be
seen under https://testing.libreswan.org/f30/



> I've been confused with the KVM_OS / KVM_VARIANT_OS variables before
> > especially in the testing.libreswan master/slave setup.
>
> Correct naming would be KVM_HOST_OS and KVM_GUEST_OS.
>
>
> --
> Tuomo Soini <tis at foobar.fi>
> Foobar Linux services
> +358 40 5240030
> Foobar Oy <https://foobar.fi/>
> _______________________________________________
> Swan-dev mailing list
> Swan-dev at lists.libreswan.org
> https://lists.libreswan.org/mailman/listinfo/swan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libreswan.org/pipermail/swan-dev/attachments/20190806/b68537d7/attachment-0001.html>

From andrew.cagney at gmail.com  Tue Aug  6 13:22:10 2019
From: andrew.cagney at gmail.com (Andrew Cagney)
Date: Tue, 6 Aug 2019 09:22:10 -0400
Subject: [Swan-dev] What should PLUTO_MY_CLIENT_NET contain?
Message-ID: <CAJeAr6upMdY4vcdw=icy8WUJRefmX4JnJBVgy3F-0CrT+ksHaQ@mail.gmail.com>

I'm looking at this code:

      jam(buf, "PLUTO_MY_CLIENT='");
        jam_subnet(buf, &sr->this.client);
        jam(buf, "' ");

        jam(buf, "PLUTO_MY_CLIENT_NET='");
        ta = subnet_endpoint(&sr->this.client);
        jam_address(buf, &ta);
        jam(buf, "' ");

        jam(buf, "PLUTO_MY_CLIENT_MASK='");
        ta = subnet_mask(&sr->this.client);
        jam_address(buf, &ta);
        jam(buf, "' ");

If you go by the names you'd think that an ip_subnet contained
network-prefix + mask-bits, but based on how it is used, it can contain

     NETWORK_PREFIX+HOST_IDENTIFIER : PORT / MASK-BITS

which means in the above, PLUTO_MY_CLIENT_NET= is set to
NETWORK_PREFIX+HOST_IDENTIFIER.

(in the old code subnet_endpoint() was called networkof() giving the
impression that just the NETWORK_PREFIX was being returned, but it wasn't).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libreswan.org/pipermail/swan-dev/attachments/20190806/faaed797/attachment.html>

From paul at nohats.ca  Tue Aug  6 14:16:15 2019
From: paul at nohats.ca (Paul Wouters)
Date: Tue, 6 Aug 2019 10:16:15 -0400 (EDT)
Subject: [Swan-dev] What should PLUTO_MY_CLIENT_NET contain?
In-Reply-To: <CAJeAr6upMdY4vcdw=icy8WUJRefmX4JnJBVgy3F-0CrT+ksHaQ@mail.gmail.com>
References: <CAJeAr6upMdY4vcdw=icy8WUJRefmX4JnJBVgy3F-0CrT+ksHaQ@mail.gmail.com>
Message-ID: <alpine.LRH.2.21.1908061014030.20832@bofh.nohats.ca>

On Tue, 6 Aug 2019, Andrew Cagney wrote:

Examples filled in for my vpn.nohats.ca connection:

> I'm looking at this code:??? ? ?jam(buf, "PLUTO_MY_CLIENT='");
> ? ? ? ? jam_subnet(buf, &sr->this.client);
> ? ? ? ? jam(buf, "' ");

PLUTO_MY_CLIENT='193.111.228.70/32'

> ? ? ? ? jam(buf, "PLUTO_MY_CLIENT_NET='");
> ? ? ? ? ta = subnet_endpoint(&sr->this.client);
> ? ? ? ? jam_address(buf, &ta);
> ? ? ? ? jam(buf, "' ");

PLUTO_MY_CLIENT_NET='193.111.228.70'

> ? ? ? ? jam(buf, "PLUTO_MY_CLIENT_MASK='");
> ? ? ? ? ta = subnet_mask(&sr->this.client);
> ? ? ? ? jam_address(buf, &ta);
> ? ? ? ? jam(buf, "' ");

PLUTO_MY_CLIENT_MASK='255.255.255.255'

> If you go by the names you'd think that an ip_subnet contained network-prefix?+ mask-bits, but based on how it is used, it can contain
> 
> ? ? ?NETWORK_PREFIX+HOST_IDENTIFIER : PORT / MASK-BITS
> 
> which means in the above, PLUTO_MY_CLIENT_NET= is set to NETWORK_PREFIX+HOST_IDENTIFIER.

That's not what seems to happen. Anyway, we should have named
PLUTO_MY_CLIENT PLUTO_MY_CLIENT_CIDR or something, but we cannot
rename anything or put different content in any of these without
breaking every single custom updown script out there.

Paul

From andrew.cagney at gmail.com  Tue Aug  6 14:40:33 2019
From: andrew.cagney at gmail.com (Andrew Cagney)
Date: Tue, 6 Aug 2019 10:40:33 -0400
Subject: [Swan-dev] heads up: KVM_OS->KVM_GUEST_OS;
	testing/libvirt/fedora30.[mk.ks}->testing/libvirt/f30.{mk.ks}
Message-ID: <CAJeAr6tJDwuFuh3eztqMD8t6-+hKPkACgaTBwzNsrPOsqGzPRg@mail.gmail.com>

If you've something like:

  KVM_OS=fedora30

set in Makefile.inc.local then you should add a second line:

  KVM_GUEST_OS=f30

I'm going to push a change that:

renames KVM_OS to KVM_GUEST_OS
  per other discussion so what this is is clarified

renames testing/libvirt/fedora30.[mk.ks} to testing/libvirt/f30.{mk.ks}
  (sitting in my queue since forever) so that things like:
      KVM_PREFIXES = ${KVM_GUEST_OS}a. $(KVM_GUEST_OS)b.
  can work - with fedora{28,30} the was too long

provided the above line is added things should continue as normal.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libreswan.org/pipermail/swan-dev/attachments/20190806/8bca3053/attachment.html>

From antony at phenome.org  Tue Aug  6 18:51:47 2019
From: antony at phenome.org (Antony Antony)
Date: Tue, 6 Aug 2019 20:51:47 +0200
Subject: [Swan-dev] heads up: KVM_OS->KVM_GUEST_OS;
 testing/libvirt/fedora30.[mk.ks}->testing/libvirt/f30.{mk.ks}
In-Reply-To: <CAJeAr6tJDwuFuh3eztqMD8t6-+hKPkACgaTBwzNsrPOsqGzPRg@mail.gmail.com>
References: <CAJeAr6tJDwuFuh3eztqMD8t6-+hKPkACgaTBwzNsrPOsqGzPRg@mail.gmail.com>
Message-ID: <20190806185147.bynx5dmpnq54yi26@AntonyAntony.local>

On Tue, Aug 06, 2019 at 10:40:33AM -0400, Andrew Cagney wrote:
> If you've something like:
> 
>   KVM_OS=fedora30 
> 
> set in Makefile.inc.local then you should add a second line:
> 
>   KVM_GUEST_OS=f30
> 
> I'm going to push a change that:
> 
> renames KVM_OS to KVM_GUEST_OS
>   per other discussion so what this is is clarified 
> 
> renames testing/libvirt/fedora30.[mk.ks} to testing/libvirt/f30.{mk.ks}
>   (sitting in my queue since forever) so that things like:
>       KVM_PREFIXES = ${KVM_GUEST_OS}a. $(KVM_GUEST_OS)b.
>   can work - with fedora{28,30} the was too long

Here's my two cents on shorter file names.
keep fedora30{mk,ks} file names. I fear increased complexity with shorter 
names together with with multiple working directories and multiple Fedora 
releases f28 and f30 in flight.

I a have a feeling shorter names would be more confusing in an advanced use 
case. I am noticing complications when switching between releases, and with 
other VMs on the same host. I wonder  what else will change due to 
shortening of filename names.

try to look at the full picture. This change would affect kvm base-domain 
and build domain names too?
Then the base-domain name may become overly cryptic.

e.g.
I have in one directory:
KVM_PREFIXES=f30w1. f30w2. f30w3. f30w4. f30w5. f30w6. f30w7. f30w8. f30w9. 
f30w10. f30w11. f30w12. f30w13. f30w14. f30w15. f30w16. f30w17. f30w18. 
f30w19. f30w20. f30w21. f30w22.

from "virsh list --all"
 -     f30w1.swanfedora28base   shut off
 -     f28w1.build              shut off

and due to a second working directory where I switch fedora28 fedora30 
 -     t1.swanfedora28base      shut off
 -     f30w1.swanfedora30base   shut off
 -     t1.build                 shut off


I feel after this change it could look like
 f30w1.swanf30base   shut off
 f30w1.build
 t1.swanf28base      shut off
 t1.swanf30base      shut off
 t1.build

VM name shortening would also increase confusion where I have other KVM 
guests on the same host. Multiple working directories and distribution is 
great.  And it comes with complexities. 

At the end naming is a matter of taste.

-antony

From antony at phenome.org  Tue Aug  6 19:19:09 2019
From: antony at phenome.org (Antony Antony)
Date: Tue, 6 Aug 2019 21:19:09 +0200
Subject: [Swan-dev] 474b105c2c KVM_PACKAGE_UPGRADE Re: [Swan-commit]
In-Reply-To: <45rC4Z47myz7WvjR@vault.libreswan.fi>
References: <45rC4Z47myz7WvjR@vault.libreswan.fi>
Message-ID: <20190806191827.zh326etkrk7wv5qo@AntonyAntony.local>

I noticed this commit a few days ago, when it broke some hacks I use!  That 
is another story. 

My guess is that 474b105c2c is trying to pin some packages such as kernel 
or/and strongswan.
If that is the case I would prefer /etc/dnf/dnf.conf way of excluding than 
$(KVM_PACKAGE_UPGRADE) alone, otherwise  dnf update and daily install would 
conflict.

we pinned kernel a while ago, and decided against it.  
https://github.com/libreswan/libreswan/commit/5302808c6cc1b998cc825353cc70ddfdfe300411

We go back and forth on pinning kernel other packages.
https://github.com/libreswan/libreswan/commit/0eaf13a7c0600d31ab464c0fc735cce074b07c47

if we want pin packages again prefered method is /etc/dnf/dnf.conf

-antony



On Sat, Jul 20, 2019 at 02:49:02AM +0000, Andrew Cagney wrote:
> New commits:
> commit 474b105c2c0acba855ed8554fd042158323db3fd
> Author: Andrew Cagney <cagney at gnu.org>
> Date:   Fri Jul 19 14:51:49 2019 -0400
> 
>     testing: add $(KVM_KERNEL_VERSION) and $(KVM_KERNEL_PACKAGES)
>     
>     So kernel version can be controlled a little.
>     (default is latest)
>     
>     Update fedora28.mk and fedora30.mk, but delete fedora29.mk.
> 

From paul at nohats.ca  Tue Aug  6 19:25:16 2019
From: paul at nohats.ca (Paul Wouters)
Date: Tue, 6 Aug 2019 15:25:16 -0400 (EDT)
Subject: [Swan-dev] 474b105c2c KVM_PACKAGE_UPGRADE Re: [Swan-commit]
In-Reply-To: <20190806191827.zh326etkrk7wv5qo@AntonyAntony.local>
References: <45rC4Z47myz7WvjR@vault.libreswan.fi>
 <20190806191827.zh326etkrk7wv5qo@AntonyAntony.local>
Message-ID: <alpine.LRH.2.21.1908061524320.7889@bofh.nohats.ca>

On Tue, 6 Aug 2019, Antony Antony wrote:

> My guess is that 474b105c2c is trying to pin some packages such as kernel
> or/and strongswan.
> If that is the case I would prefer /etc/dnf/dnf.conf way of excluding than
> $(KVM_PACKAGE_UPGRADE) alone, otherwise  dnf update and daily install would
> conflict.

We should not pin strongswan or the kernel? Now that we are on "bleeding
edge", I think we can take the bleeding when it happens.

Paul

From andrew.cagney at gmail.com  Wed Aug  7 02:45:34 2019
From: andrew.cagney at gmail.com (Andrew Cagney)
Date: Tue, 6 Aug 2019 22:45:34 -0400
Subject: [Swan-dev] 474b105c2c KVM_PACKAGE_UPGRADE Re: [Swan-commit]
In-Reply-To: <alpine.LRH.2.21.1908061524320.7889@bofh.nohats.ca>
References: <45rC4Z47myz7WvjR@vault.libreswan.fi>
 <20190806191827.zh326etkrk7wv5qo@AntonyAntony.local>
 <alpine.LRH.2.21.1908061524320.7889@bofh.nohats.ca>
Message-ID: <CAJeAr6uvLf_+tTaHDH+RFi=S=JYkGDTvi-ve9qKLd8gcRxwJFA@mail.gmail.com>

On Tue, 6 Aug 2019 at 15:25, Paul Wouters <paul at nohats.ca> wrote:

> On Tue, 6 Aug 2019, Antony Antony wrote:
>
> > My guess is that 474b105c2c is trying to pin some packages such as kernel
> > or/and strongswan.
> > If that is the case I would prefer /etc/dnf/dnf.conf way of excluding
> than
> > $(KVM_PACKAGE_UPGRADE) alone, otherwise  dnf update and daily install
> would
> > conflict.
>
> We should not pin strongswan or the kernel? Now that we are on "bleeding
> edge", I think we can take the bleeding when it happens.
>
>
Having just tried to go through old releases and reproduce test results,
I'm forced to conclude that agreeing to unpinning strongswan and the kernel
was a stupid foolish mistake.

For instance, I can't even reproduce f28's results (yet alone anything
earlier).  This is because a kernel that didn't even exist at the time f28
was released gets installed on the test VMs.  Instead, for the test VMs we
need to ensure that what we tested the release against an always (within a
few limitations) be reproduced.

Andrew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libreswan.org/pipermail/swan-dev/attachments/20190806/81aec1ab/attachment-0001.html>

From andrew.cagney at gmail.com  Wed Aug  7 16:27:10 2019
From: andrew.cagney at gmail.com (Andrew Cagney)
Date: Wed, 7 Aug 2019 12:27:10 -0400
Subject: [Swan-dev] What should PLUTO_MY_CLIENT_NET contain?
In-Reply-To: <alpine.LRH.2.21.1908061014030.20832@bofh.nohats.ca>
References: <CAJeAr6upMdY4vcdw=icy8WUJRefmX4JnJBVgy3F-0CrT+ksHaQ@mail.gmail.com>
 <alpine.LRH.2.21.1908061014030.20832@bofh.nohats.ca>
Message-ID: <CAJeAr6upGa86fSZCTVFXT5sUXcA=q0AiMmmc7W-emoeJf-zLkg@mail.gmail.com>

On Tue, 6 Aug 2019 at 10:16, Paul Wouters <paul at nohats.ca> wrote:
>
> On Tue, 6 Aug 2019, Andrew Cagney wrote:
>
> Examples filled in for my vpn.nohats.ca connection:
>

Yea, 193.111.228.70/32 is the easy case since PREFIX+HOST ==
193.111.228.70; same with subnets like 193.111.228.0/24.

But then there's :PORT:

    193.111.228.70/32:555

Both pfkey_raw_eroute() and eroute contain code that checks for a
non-zero :PORT so it can happen.  The code below does not include it
(neither jam_subnet() / subnettot() nor jam_address() / addrtot()
include the port, even when it is non-zero).

Finally there's:

    193.111.228.70/24:555

the subnet parser seems to allow it but I don't know if it is rejected
later on.  If it is, then ..._NET=193.111.228.70 is added.


> > I'm looking at this code:       jam(buf, "PLUTO_MY_CLIENT='");
> >         jam_subnet(buf, &sr->this.client);
> >         jam(buf, "' ");
>
> PLUTO_MY_CLIENT='193.111.228.70/32'
>
> >         jam(buf, "PLUTO_MY_CLIENT_NET='");
> >         ta = subnet_endpoint(&sr->this.client);
> >         jam_address(buf, &ta);
> >         jam(buf, "' ");
>
> PLUTO_MY_CLIENT_NET='193.111.228.70'
>
> >         jam(buf, "PLUTO_MY_CLIENT_MASK='");
> >         ta = subnet_mask(&sr->this.client);
> >         jam_address(buf, &ta);
> >         jam(buf, "' ");
>
> PLUTO_MY_CLIENT_MASK='255.255.255.255'
>
> > If you go by the names you'd think that an ip_subnet contained network-prefix + mask-bits, but based on how it is used, it can contain
> >
> >      NETWORK_PREFIX+HOST_IDENTIFIER : PORT / MASK-BITS
> >
> > which means in the above, PLUTO_MY_CLIENT_NET= is set to NETWORK_PREFIX+HOST_IDENTIFIER.
>
> That's not what seems to happen. Anyway, we should have named
> PLUTO_MY_CLIENT PLUTO_MY_CLIENT_CIDR or something, but we cannot
> rename anything or put different content in any of these without
> breaking every single custom updown script out there.
>
> Paul

From andrew.cagney at gmail.com  Thu Aug  8 14:48:17 2019
From: andrew.cagney at gmail.com (Andrew Cagney)
Date: Thu, 8 Aug 2019 10:48:17 -0400
Subject: [Swan-dev] heads up: switching KVM_GUEST_OS to f30
Message-ID: <CAJeAr6vVs+6j3nwD42C0AY030tW2tGF77z38=sPazifzqD46eQ@mail.gmail.com>

While the f30 failures are around 130 -
https://testing.libreswan.org/f30/ - the changes to do this have
caused the f28 failures to drop to 160 -
https://testing.libreswan.org/ - which makes still having the default
guest os set to f28 somewhat pointless.

From andrew.cagney at gmail.com  Thu Aug  8 15:36:38 2019
From: andrew.cagney at gmail.com (Andrew Cagney)
Date: Thu, 8 Aug 2019 11:36:38 -0400
Subject: [Swan-dev] heads up: KVM_OS->KVM_GUEST_OS;
	testing/libvirt/fedora30.[mk.ks}->testing/libvirt/f30.{mk.ks}
In-Reply-To: <20190806185147.bynx5dmpnq54yi26@AntonyAntony.local>
References: <CAJeAr6tJDwuFuh3eztqMD8t6-+hKPkACgaTBwzNsrPOsqGzPRg@mail.gmail.com>
 <20190806185147.bynx5dmpnq54yi26@AntonyAntony.local>
Message-ID: <CAJeAr6spb0pQ+6B8CXDL+THg4T5RDVk2PhVapwvmHuG1=T1r6A@mail.gmail.com>

On Tue, 6 Aug 2019 at 14:51, Antony Antony <antony at phenome.org> wrote:
>
> On Tue, Aug 06, 2019 at 10:40:33AM -0400, Andrew Cagney wrote:
> > If you've something like:
> >
> >   KVM_OS=fedora30
> >
> > set in Makefile.inc.local then you should add a second line:
> >
> >   KVM_GUEST_OS=f30
> >
> > I'm going to push a change that:
> >
> > renames KVM_OS to KVM_GUEST_OS
> >   per other discussion so what this is is clarified
> >
> > renames testing/libvirt/fedora30.[mk.ks} to testing/libvirt/f30.{mk.ks}
> >   (sitting in my queue since forever) so that things like:
> >       KVM_PREFIXES = ${KVM_GUEST_OS}a. $(KVM_GUEST_OS)b.
> >   can work - with fedora{28,30} the was too long
>
> Here's my two cents on shorter file names.
> keep fedora30{mk,ks} file names. I fear increased complexity with shorter
> names together with with multiple working directories and multiple Fedora
> releases f28 and f30 in flight.
>
> I a have a feeling shorter names would be more confusing in an advanced use
> case. I am noticing complications when switching between releases, and with
> other VMs on the same host. I wonder  what else will change due to
> shortening of filename names.
>
> try to look at the full picture. This change would affect kvm base-domain
> and build domain names too?
> Then the base-domain name may become overly cryptic.
>
> e.g.
> I have in one directory:
> KVM_PREFIXES=f30w1. f30w2. f30w3. f30w4. f30w5. f30w6. f30w7. f30w8. f30w9.
> f30w10. f30w11. f30w12. f30w13. f30w14. f30w15. f30w16. f30w17. f30w18.
> f30w19. f30w20. f30w21. f30w22.

Which, with the shorter name, can finally be written as:

KVM_PREFIXES=$(KVM_GUEST_OS)w1. $(KVM_GUEST_OS)w2. ...

and flipping between OSs becomes a matter of setting KVM_GUEST_OS=...
and flipping won't keep triggering full KVM rebuilds

It will also let us have KVM_PREFIXES=$(KVM_GUEST_OS). as the default
at some point.

> from "virsh list --all"
>  -     f30w1.swanfedora28base   shut off
>  -     f28w1.build              shut off
>
> and due to a second working directory where I switch fedora28 fedora30
>  -     t1.swanfedora28base      shut off
>  -     f30w1.swanfedora30base   shut off
>  -     t1.build                 shut off
>
> I feel after this change it could look like
>  f30w1.swanf30base   shut off
>  f30w1.build
>  t1.swanf28base      shut off
>  t1.swanf30base      shut off
>  t1.build

Right.  I'm not sure how this is a problem though.
As an aside I'd like to shorten *swan*base, $(KVM_FIRST_PREFIX)base
comes to mind, later.

> VM name shortening would also increase confusion where I have other KVM
> guests on the same host. Multiple working directories and distribution is
> great.  And it comes with complexities.

Right, we've already got that problem with t1. and with east, west,
.... when no prefix is specified.  We could prepend "lsw." to
everything but then we run slam bang into the network interface's
hard-wired name-length limit (we'd have to claw back some characters
from the 192_.. suffix).

> At the end naming is a matter of taste.
>
> -antony

From andrew.cagney at gmail.com  Tue Aug 13 17:52:34 2019
From: andrew.cagney at gmail.com (Andrew Cagney)
Date: Tue, 13 Aug 2019 13:52:34 -0400
Subject: [Swan-dev] testing: remove set -x from web-targets.mk
In-Reply-To: <20190729205042.grya6b5epzzmpdmm@AntonyAntony.local>
References: <45xyvy3VsPz7VyXP@vault.libreswan.fi>
 <CAJeAr6sn=cRBrEL6ekzAcCZOMOOudgioQ71siTtLE9pojFdspw@mail.gmail.com>
 <CAJeAr6suhPeNLbCtaOb1=oBtmo9k3pRvu1RGeQa0nkisdNhgEg@mail.gmail.com>
 <20190729182700.kp3obo6te77oqklc@AntonyAntony.local>
 <CAJeAr6v86CS7X8sy=GNJpt7hjprinJmFzQEsDREJ+E7p7eNsHQ@mail.gmail.com>
 <20190729195825.3u4dcmgz6h42fcdy@AntonyAntony.local>
 <CAJeAr6u55XDKX2-EvoiGZ4PP7JXYVz1gUEuLh4SXQ6kKGqCPaw@mail.gmail.com>
 <20190729205042.grya6b5epzzmpdmm@AntonyAntony.local>
Message-ID: <CAJeAr6t_ivmk+OUuMq83pgTqnh1s3Do7yBydHR9ezKQ9iLS1ag@mail.gmail.com>

I locally put back the set -x; and added a lot more.  This turned up:

real    0m34.006s
user    0m0.220s
sys    0m2.488s
+ test -d .git -o -f .git
+ git rev-parse --abbrev-ref HEAD

It explain why 'make' on a KVM seemed to hang at the start.  Like I
said, the reason I orginally added 'set -x' was so that we'd be
immediately alerted to problems like this.


On Mon, 29 Jul 2019 at 16:50, Antony Antony <antony at phenome.org> wrote:
>
> On Mon, Jul 29, 2019 at 04:20:24PM -0400, Andrew Cagney wrote:
> > On Mon, 29 Jul 2019 at 15:58, Antony Antony <antony at phenome.org> wrote:
> >
> > > On Mon, Jul 29, 2019 at 03:18:03PM -0400, Andrew Cagney wrote:
> > > > On Mon, 29 Jul 2019 at 14:27, Antony Antony <antony at phenome.org> wrote:
> > > >
> > > > > On Mon, Jul 29, 2019 at 11:31:30AM -0400, Andrew Cagney wrote:
> > > > > > On Mon, 29 Jul 2019 at 08:38, Andrew Cagney <andrew.cagney at gmail.com
> > > >
> > > > > wrote:
> > > > > >
> > > > > > > These make variables should only be expanded when web pages are
> > > > > enabled?
> > > > > > > Per the comment:
> > > > > > >
> > > > > > > # shortcuts to use when web is enabled, set up to evaluate once as
> > > > > > > # they can be a little expensive.  These make variable can only be
> > > > > > > # used inside of WEB_ENABLED blocks.
> > > > > > >
> > > > > >
> > > > > > In fact it was working.  Here's what happens in my tree using the
> > > commit
> > > > > > just before the below:
> > > > > >
> > > > > > $ v=$(gmake showversion)
> > > > > > $ echo $v
> > > > > > 3.28-506-g423d0e04bd-HEAD
> > > > > > $ v=$(make showversion LSW_WEBDIR=..)
> > > > > > + cd .
> > > > > > + git show --no-patch --format=%H HEAD
> > > > > > + testing/web/gime-git-description.sh .
> > > > > > $ echo $v
> > > > > > 3.28-506-g423d0e04bd-HEAD
> > > > > > $
> > > > > >
> > > > > > so I'm stumped as to why the below behaviour occurred.
> > > > >
> > > > > I added LSW_WEBDIR this month, this explains why I started to see
> > > change
> > > > > in
> > > > > behaviour of 'make showversion'. the commit that added set -x is old
> > > and I
> > > > > wondered why didn't I notice set -x before. Thanks for the information.
> > > > >
> > > > > 'make showversion' output should not change based on other variables
> > > are
> > > > > set
> > > > > or not! It is used by scripts.
> > > > >
> > > > > If you still want to see verbose, we a need different way without
> > > changing
> > > > > output of 'make showversion' or other targets used by scripts.
> > > > >
> > > > >
> > > > The output as in stdout and, as illustrated above, can be captured in a
> > > > shell variable, is unchanged.
> > > > Can you be more specific w.r.t., the problem you're seeing?
> > >
> > > two issues. it is annoying to see extra output when I expect short one
> > > line
> > > output.
> > > If we go down this path simple commands give overly complex output to read.
> > > This also affected make showdebversion, make showrpmversion..
> > >
> > >
> > Personally, I'd only use those and related packaging targets on a clean
> > tree with Makefile.inc.local empty.
>
> well we clearly have different use cases! I use those commands very often.
> It is annoying to see stdout + stderr  behave different based on other
> variables in Makefile.inc.local.
>
> here is one case.
> I often login to servers, both mine and Paul's where git libreswan is
> running, then look at pluto's log file for version and run
> make showveresion to compare. sometimes pluto --version also. Then I know
> which source tree is installed and running.
>
> These machines typically have multiple source directories lying around.
> Unless you compare the running version you would be in wrong source tree.
> debugging from wrong source tree.
>
> If paul is wokring on a directory I want to leave that alone, during my
> quick test.
>
> make showversion is used! when debugging and developing.
> More than once I have noticed difference between running version and where I
> start debugging, or other points me.
>
> > second, I am using subprocess.getoutput() python function. which mixes
> > > stdout and stderror! I could replace this, the first reason still stands!
> > >
> > >
> > That seems very frail.
>
> from old python days! with 3.6 and later it is easier to split stdout and
> stderror.
>
> >
> >
> > -antony
> > >
> > > > >
> > > > > >
> > > > > > Andrew
> > > > > >
> > > > > > At one point they were very expensive, unconditionally expanded, and
> > > > > > > repeatedly making everything slow.  Hence their verbosity - so we
> > > can
> > > > > see
> > > > > > > if the bug has crept back in
> > > > > > > (from memory, most of the expense was in computing 'dirty' which
> > > is why
> > > > > > > the web page code doesn't do that)
> > > > > > >
> > > > > > > Andrew
> > > > > > >
> > > > > > > ---------- Forwarded message ---------
> > > > > > > From: Antony Antony <antony at vault.libreswan.fi>
> > > > > > > Date: Mon, 29 Jul 2019 at 08:01
> > > > > > > Subject: [Swan-commit] Changes to ref refs/heads/master
> > > > > > > To: <swan-commit at lists.libreswan.org>
> > > > > > >
> > > > > > >
> > > > > > > New commits:
> > > > > > > commit 367867e7124949f029c5bf0b3b0f527384770bad
> > > > > > > Author: Antony Antony <antony at phenome.org>
> > > > > > > Date:   Sun Jul 21 11:08:10 2019 +0000
> > > > > > >
> > > > > > >     testing: remove set -x from web-targets.mk
> > > > > > >
> > > > > > >     after:
> > > > > > >     make showversion
> > > > > > >     3.28-504-g5113b92b34-master
> > > > > > >
> > > > > > >     before:
> > > > > > >     make showversion
> > > > > > >     + cd .
> > > > > > >     + git show --no-patch --format=%H HEAD
> > > > > > >     + testing/web/gime-git-description.sh .
> > > > > > >     3.28-504-g5113b92b34-dirty-master
> > > > > > >
> > > > > > > _______________________________________________
> > > > > > > Swan-commit mailing list
> > > > > > > Swan-commit at lists.libreswan.org
> > > > > > > https://lists.libreswan.org/mailman/listinfo/swan-commit
> > > > > > >
> > > > >
> > > > > > _______________________________________________
> > > > > > Swan-dev mailing list
> > > > > > Swan-dev at lists.libreswan.org
> > > > > > https://lists.libreswan.org/mailman/listinfo/swan-dev
> > > > >
> > > > >
> > >

From andrew.cagney at gmail.com  Thu Aug 15 17:32:56 2019
From: andrew.cagney at gmail.com (Andrew Cagney)
Date: Thu, 15 Aug 2019 13:32:56 -0400
Subject: [Swan-dev] testing: remove set -x from web-targets.mk
In-Reply-To: <CAJeAr6t_ivmk+OUuMq83pgTqnh1s3Do7yBydHR9ezKQ9iLS1ag@mail.gmail.com>
References: <45xyvy3VsPz7VyXP@vault.libreswan.fi>
 <CAJeAr6sn=cRBrEL6ekzAcCZOMOOudgioQ71siTtLE9pojFdspw@mail.gmail.com>
 <CAJeAr6suhPeNLbCtaOb1=oBtmo9k3pRvu1RGeQa0nkisdNhgEg@mail.gmail.com>
 <20190729182700.kp3obo6te77oqklc@AntonyAntony.local>
 <CAJeAr6v86CS7X8sy=GNJpt7hjprinJmFzQEsDREJ+E7p7eNsHQ@mail.gmail.com>
 <20190729195825.3u4dcmgz6h42fcdy@AntonyAntony.local>
 <CAJeAr6u55XDKX2-EvoiGZ4PP7JXYVz1gUEuLh4SXQ6kKGqCPaw@mail.gmail.com>
 <20190729205042.grya6b5epzzmpdmm@AntonyAntony.local>
 <CAJeAr6t_ivmk+OUuMq83pgTqnh1s3Do7yBydHR9ezKQ9iLS1ag@mail.gmail.com>
Message-ID: <CAJeAr6suKk_y0MyHF=N_9YMGOK3J9U+joYwf7L_0T-oURLegnw@mail.gmail.com>

On Tue, 13 Aug 2019 at 13:52, Andrew Cagney <andrew.cagney at gmail.com> wrote:
>
> I locally put back the set -x; and added a lot more.  This turned up:
>
> real    0m34.006s
> user    0m0.220s
> sys    0m2.488s
> + test -d .git -o -f .git
> + git rev-parse --abbrev-ref HEAD
>
> It explain why 'make' on a KVM seemed to hang at the start.  Like I
> said, the reason I orginally added 'set -x' was so that we'd be
> immediately alerted to problems like this.

This is the cause:

BRANCH = $(shell set -x ; test -d .git -o -f .git && (time git
rev-parse --abbrev-ref HEAD || echo ''))
TRAVIS_BANCH = $(call W1, $(BRANCH),'')
ifeq ($(TRAVIS_BANCH), travis)
    DISTRO =  $(call W2, $(BRANCH),fedora)
    DISTRO_REL = $(call W3, $(BRANCH),27)
endif

given it's wasting time on every build, any reason to not simply disable it?

>
> On Mon, 29 Jul 2019 at 16:50, Antony Antony <antony at phenome.org> wrote:
> >
> > On Mon, Jul 29, 2019 at 04:20:24PM -0400, Andrew Cagney wrote:
> > > On Mon, 29 Jul 2019 at 15:58, Antony Antony <antony at phenome.org> wrote:
> > >
> > > > On Mon, Jul 29, 2019 at 03:18:03PM -0400, Andrew Cagney wrote:
> > > > > On Mon, 29 Jul 2019 at 14:27, Antony Antony <antony at phenome.org> wrote:
> > > > >
> > > > > > On Mon, Jul 29, 2019 at 11:31:30AM -0400, Andrew Cagney wrote:
> > > > > > > On Mon, 29 Jul 2019 at 08:38, Andrew Cagney <andrew.cagney at gmail.com
> > > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > > These make variables should only be expanded when web pages are
> > > > > > enabled?
> > > > > > > > Per the comment:
> > > > > > > >
> > > > > > > > # shortcuts to use when web is enabled, set up to evaluate once as
> > > > > > > > # they can be a little expensive.  These make variable can only be
> > > > > > > > # used inside of WEB_ENABLED blocks.
> > > > > > > >
> > > > > > >
> > > > > > > In fact it was working.  Here's what happens in my tree using the
> > > > commit
> > > > > > > just before the below:
> > > > > > >
> > > > > > > $ v=$(gmake showversion)
> > > > > > > $ echo $v
> > > > > > > 3.28-506-g423d0e04bd-HEAD
> > > > > > > $ v=$(make showversion LSW_WEBDIR=..)
> > > > > > > + cd .
> > > > > > > + git show --no-patch --format=%H HEAD
> > > > > > > + testing/web/gime-git-description.sh .
> > > > > > > $ echo $v
> > > > > > > 3.28-506-g423d0e04bd-HEAD
> > > > > > > $
> > > > > > >
> > > > > > > so I'm stumped as to why the below behaviour occurred.
> > > > > >
> > > > > > I added LSW_WEBDIR this month, this explains why I started to see
> > > > change
> > > > > > in
> > > > > > behaviour of 'make showversion'. the commit that added set -x is old
> > > > and I
> > > > > > wondered why didn't I notice set -x before. Thanks for the information.
> > > > > >
> > > > > > 'make showversion' output should not change based on other variables
> > > > are
> > > > > > set
> > > > > > or not! It is used by scripts.
> > > > > >
> > > > > > If you still want to see verbose, we a need different way without
> > > > changing
> > > > > > output of 'make showversion' or other targets used by scripts.
> > > > > >
> > > > > >
> > > > > The output as in stdout and, as illustrated above, can be captured in a
> > > > > shell variable, is unchanged.
> > > > > Can you be more specific w.r.t., the problem you're seeing?
> > > >
> > > > two issues. it is annoying to see extra output when I expect short one
> > > > line
> > > > output.
> > > > If we go down this path simple commands give overly complex output to read.
> > > > This also affected make showdebversion, make showrpmversion..
> > > >
> > > >
> > > Personally, I'd only use those and related packaging targets on a clean
> > > tree with Makefile.inc.local empty.
> >
> > well we clearly have different use cases! I use those commands very often.
> > It is annoying to see stdout + stderr  behave different based on other
> > variables in Makefile.inc.local.
> >
> > here is one case.
> > I often login to servers, both mine and Paul's where git libreswan is
> > running, then look at pluto's log file for version and run
> > make showveresion to compare. sometimes pluto --version also. Then I know
> > which source tree is installed and running.
> >
> > These machines typically have multiple source directories lying around.
> > Unless you compare the running version you would be in wrong source tree.
> > debugging from wrong source tree.
> >
> > If paul is wokring on a directory I want to leave that alone, during my
> > quick test.
> >
> > make showversion is used! when debugging and developing.
> > More than once I have noticed difference between running version and where I
> > start debugging, or other points me.
> >
> > > second, I am using subprocess.getoutput() python function. which mixes
> > > > stdout and stderror! I could replace this, the first reason still stands!
> > > >
> > > >
> > > That seems very frail.
> >
> > from old python days! with 3.6 and later it is easier to split stdout and
> > stderror.
> >
> > >
> > >
> > > -antony
> > > >
> > > > > >
> > > > > > >
> > > > > > > Andrew
> > > > > > >
> > > > > > > At one point they were very expensive, unconditionally expanded, and
> > > > > > > > repeatedly making everything slow.  Hence their verbosity - so we
> > > > can
> > > > > > see
> > > > > > > > if the bug has crept back in
> > > > > > > > (from memory, most of the expense was in computing 'dirty' which
> > > > is why
> > > > > > > > the web page code doesn't do that)
> > > > > > > >
> > > > > > > > Andrew
> > > > > > > >
> > > > > > > > ---------- Forwarded message ---------
> > > > > > > > From: Antony Antony <antony at vault.libreswan.fi>
> > > > > > > > Date: Mon, 29 Jul 2019 at 08:01
> > > > > > > > Subject: [Swan-commit] Changes to ref refs/heads/master
> > > > > > > > To: <swan-commit at lists.libreswan.org>
> > > > > > > >
> > > > > > > >
> > > > > > > > New commits:
> > > > > > > > commit 367867e7124949f029c5bf0b3b0f527384770bad
> > > > > > > > Author: Antony Antony <antony at phenome.org>
> > > > > > > > Date:   Sun Jul 21 11:08:10 2019 +0000
> > > > > > > >
> > > > > > > >     testing: remove set -x from web-targets.mk
> > > > > > > >
> > > > > > > >     after:
> > > > > > > >     make showversion
> > > > > > > >     3.28-504-g5113b92b34-master
> > > > > > > >
> > > > > > > >     before:
> > > > > > > >     make showversion
> > > > > > > >     + cd .
> > > > > > > >     + git show --no-patch --format=%H HEAD
> > > > > > > >     + testing/web/gime-git-description.sh .
> > > > > > > >     3.28-504-g5113b92b34-dirty-master
> > > > > > > >
> > > > > > > > _______________________________________________
> > > > > > > > Swan-commit mailing list
> > > > > > > > Swan-commit at lists.libreswan.org
> > > > > > > > https://lists.libreswan.org/mailman/listinfo/swan-commit
> > > > > > > >
> > > > > >
> > > > > > > _______________________________________________
> > > > > > > Swan-dev mailing list
> > > > > > > Swan-dev at lists.libreswan.org
> > > > > > > https://lists.libreswan.org/mailman/listinfo/swan-dev
> > > > > >
> > > > > >
> > > >

From andrew.cagney at gmail.com  Fri Aug 16 22:15:55 2019
From: andrew.cagney at gmail.com (Andrew Cagney)
Date: Fri, 16 Aug 2019 18:15:55 -0400
Subject: [Swan-dev] pointlessly building the KLIPS kernel module on the KVMs
Message-ID: <CAJeAr6ujMbeT+bzYPii-+dO3ge6V_b3UUUWdk3ckYa5V9psGfw@mail.gmail.com>

If you look through a recent test result on
https://testing.libreswan.org/ you'll notice KERNEL errors - these are
from the KLIPS module panicking.  This has problem seemingly been
known for some time but forgotten - KLIPS will panic on >=4.x kernels
(at least those shipped by fedora).

Given the code is isolated and (hopeful) treated read-only - itt
should still build on old systems.  However, I struggle to see a
reason for building it for Fedora 30, and would again like to disable
it.

(this shouldn't be confused with the klips userland code which can
continue to build).

Andrew

From hugh at mimosa.com  Sat Aug 17 14:42:35 2019
From: hugh at mimosa.com (D. Hugh Redelmeier)
Date: Sat, 17 Aug 2019 10:42:35 -0400 (EDT)
Subject: [Swan-dev] redirect: in
 emit_redirect_notification_decoded_dest() delete '-' in column 1 negating a
 bool
Message-ID: <alpine.LFD.2.21.1908171033170.22456@redeye.mimosa.com>

| commit b473d2b9be0a244672aa2934e3feaac55ad23b82
| Author: Andrew Cagney <cagney at gnu.org>
| Date:   Sat Aug 17 08:08:56 2019 -0400
| 
|     redirect: in emit_redirect_notification_decoded_dest() delete '-' in column 1 negating a bool
|     
|     Looks like a merge botch.  Worryingly, fixing this doesn't seem to
|     affect test results - it would have caused the initiator code path
|     to always fail?
|     
|     See 4e3d1805639c05e3fd00c4d175979916111476b5

Good catch!  Thanks.

Interestingly, in C, applying "-" to a bool does not change the value
if it is used as a bool.

a:		false	true
-a:		0	-1
(bool) -a:	false	true

So: no testing could find this bug since it causes no difference in
behaviour.

From andrew.cagney at gmail.com  Sat Aug 17 15:42:05 2019
From: andrew.cagney at gmail.com (Andrew Cagney)
Date: Sat, 17 Aug 2019 11:42:05 -0400
Subject: [Swan-dev] redirect: in
 emit_redirect_notification_decoded_dest() delete '-' in column 1 negating a
 bool
In-Reply-To: <alpine.LFD.2.21.1908171033170.22456@redeye.mimosa.com>
References: <alpine.LFD.2.21.1908171033170.22456@redeye.mimosa.com>
Message-ID: <CAJeAr6tX15M0dV6fJ5kQA-up6JpXhXL1-qCb+938FBqPnHaBWA@mail.gmail.com>

ah, yes, thanks for explaining that

On Sat, 17 Aug 2019 at 10:42, D. Hugh Redelmeier <hugh at mimosa.com> wrote:
>
> | commit b473d2b9be0a244672aa2934e3feaac55ad23b82
> | Author: Andrew Cagney <cagney at gnu.org>
> | Date:   Sat Aug 17 08:08:56 2019 -0400
> |
> |     redirect: in emit_redirect_notification_decoded_dest() delete '-' in column 1 negating a bool
> |
> |     Looks like a merge botch.  Worryingly, fixing this doesn't seem to
> |     affect test results - it would have caused the initiator code path
> |     to always fail?
> |
> |     See 4e3d1805639c05e3fd00c4d175979916111476b5
>
> Good catch!  Thanks.
>
> Interestingly, in C, applying "-" to a bool does not change the value
> if it is used as a bool.
>
> a:              false   true
> -a:             0       -1
> (bool) -a:      false   true
>
> So: no testing could find this bug since it causes no difference in
> behaviour.
> _______________________________________________
> Swan-dev mailing list
> Swan-dev at lists.libreswan.org
> https://lists.libreswan.org/mailman/listinfo/swan-dev

From andrew.cagney at gmail.com  Sun Aug 18 14:10:42 2019
From: andrew.cagney at gmail.com (Andrew Cagney)
Date: Sun, 18 Aug 2019 10:10:42 -0400
Subject: [Swan-dev] newoe-25-cat-3-4-way and our id=%any
Message-ID: <CAJeAr6vbJpbn46FMuav06BK0jBGesRyOyjuf-GWf=WBYL6JGVg@mail.gmail.com>

I'm puzzled by this failure:

--- MASTER/testing/pluto/newoe-25-cat-3-4-way/road.console.txt
+++ OUTPUT/testing/pluto/newoe-25-cat-3-4-way/road.console.txt
@@ -124,7 +124,7 @@
 000 "block":   policy: AUTH_NEVER+GROUP+GROUTED+REJECT+NEVER_NEGOTIATE;
 000 "block":   conn_prio: 32,32; interface: eth0; metric: 0; mtu:
unset; sa_prio:auto; sa_tfc:none;
 000 "block":   nflog-group: unset; mark: unset; vti-iface:unset;
vti-routing:no; vti-shared:no; nic-offload:no;
-000 "block":   our idtype: ID_IPV4_ADDR; our id=192.1.3.209; their
idtype: %none; their id=(none)
+000 "block":   our idtype: ID_IPV4_ADDR; our id=%any; their idtype:
%none; their id=(none)
 000 "block":   dpd: action:disabled; delay:0; timeout:0; nat-t:
encaps:no; nat_keepalive:no; ikev1_natt:both
 000 "block":   newest ISAKMP SA: #0; newest IPsec SA: #0;
 000 "clear": 192.1.3.209---192.1.3.254...%group; unrouted; eroute owner: #0

I'm not sure about this part:

if (e->id.kind == ID_NONE && !isanyaddr(&e->host_addr)) {
  e->id.kind = afi->id_addr;
  e->id.ip_addr = e->host_addr;
  e->id.isanyid = TRUE; /* used to match id=%any */ <- new line
  e->has_id_wildcards = FALSE;
}

of the change
https://github.com/libreswan/libreswan/commit/8bb225798220221396d63cb039d0f3dbb7fb739e

!isanyaddr() is true when !invalid && !any, i.e., an address was specified.

Andrew

From antony at phenome.org  Tue Aug 20 20:54:11 2019
From: antony at phenome.org (Antony Antony)
Date: Tue, 20 Aug 2019 22:54:11 +0200
Subject: [Swan-dev] kvm-install speed up run git commands on host
Message-ID: <20190820205411.guslygzlhubs6z7v@AntonyAntony.local>

Here is an idea to speedup up building inside kvm when using "kvm-install"  
from what I red on IRC this would save upto 80seconds in the build.  Testrun 
could take 4hours.

I am curious if it helps Andrew.
I am trying avoid loosing features in the name saving few seconds.

-antony
-------------- next part --------------
>From c08a27909f9bbf132c413738fd5c1485ef11cbaa Mon Sep 17 00:00:00 2001
From: Antony Antony <antony at phenome.org>
Date: Tue, 20 Aug 2019 20:41:33 +0000
Subject: [PATCH] testing: define IPSECVERSION for buildig to speed up kvm
 build


diff --git a/mk/kvm-targets.mk b/mk/kvm-targets.mk
index 5e7308f2b0..05480a9bbe 100644
--- a/mk/kvm-targets.mk
+++ b/mk/kvm-targets.mk
@@ -935,9 +935,9 @@ kvm-$(KVM_BUILD_DOMAIN)-build: \
 		| \
 		$(KVM_LOCALDIR)/$(KVM_BUILD_DOMAIN).xml
 	$(KVMSH) $(KVMSH_FLAGS) --chdir . $(KVM_BUILD_DOMAIN) 'export OBJDIR=$(KVM_OBJDIR)'
-	$(KVMSH) $(KVMSH_FLAGS) --chdir . $(KVM_BUILD_DOMAIN) 'make OBJDIR=$(KVM_OBJDIR) $(KVM_MAKEFLAGS) base'
+	$(KVMSH) $(KVMSH_FLAGS) --chdir . $(KVM_BUILD_DOMAIN) 'make IPSECVERSION=$(IPSECVERSION) OBJDIR=$(KVM_OBJDIR) $(KVM_MAKEFLAGS) base'
 ifeq ($(KVM_USE_KLIPS),true)
-	$(KVMSH) $(KVMSH_FLAGS) --chdir . $(KVM_BUILD_DOMAIN) 'make OBJDIR=$(KVM_OBJDIR) $(KVM_MAKEFLAGS) module'
+	$(KVMSH) $(KVMSH_FLAGS) --chdir . $(KVM_BUILD_DOMAIN) 'make IPSECVERSION=$(IPSECVERSION) OBJDIR=$(KVM_OBJDIR) $(KVM_MAKEFLAGS) module'
 endif
 	: install will run $(KVMSH) --shutdown $(1)
 
@@ -954,12 +954,12 @@ kvm-build: $(foreach domain, $(KVM_BUILD_DOMAIN_CLONES), uninstall-kvm-domain-$(
 
 .PHONY: kvm-$(KVM_BUILD_DOMAIN)-install
 kvm-$(KVM_BUILD_DOMAIN)-install: $(KVM_QEMUDIR_OK) kvm-$(KVM_BUILD_DOMAIN)-build | $(KVM_LOCALDIR)/$(KVM_BUILD_DOMAIN).xml
-	$(KVMSH) $(KVMSH_FLAGS) --chdir . $(KVM_BUILD_DOMAIN) 'make OBJDIR=$(KVM_OBJDIR) $(KVM_MAKEFLAGS) install-base'
+	$(KVMSH) $(KVMSH_FLAGS) --chdir . $(KVM_BUILD_DOMAIN) 'make IPSECVERSION=$(IPSECVERSION) OBJDIR=$(KVM_OBJDIR) $(KVM_MAKEFLAGS) install-base'
 ifeq ($(KVM_USE_FIPSCHECK),true)
-	$(KVMSH) $(KVMSH_FLAGS) --chdir . $(KVM_BUILD_DOMAIN) 'make OBJDIR=$(KVM_OBJDIR) $(KVM_MAKEFLAGS) install-fipshmac'
+	$(KVMSH) $(KVMSH_FLAGS) --chdir . $(KVM_BUILD_DOMAIN) 'make IPSECVERSION=$(IPSECVERSION) OBJDIR=$(KVM_OBJDIR) $(KVM_MAKEFLAGS) install-fipshmac'
 endif
 ifeq ($(KVM_USE_KLIPS),true)
-	$(KVMSH) $(KVMSH_FLAGS) --chdir . $(KVM_BUILD_DOMAIN) 'make OBJDIR=$(KVM_OBJDIR) $(KVM_MAKEFLAGS) module_install'
+	$(KVMSH) $(KVMSH_FLAGS) --chdir . $(KVM_BUILD_DOMAIN) 'make IPSECVERSION=$(IPSECVERSION) OBJDIR=$(KVM_OBJDIR) $(KVM_MAKEFLAGS) module_install'
 endif
 	$(KVMSH) $(KVMSH_FLAGS) --chdir . $(KVM_BUILD_DOMAIN) 'restorecon /usr/local/sbin /usr/local/libexec/ipsec -Rv'
 	$(KVMSH) --shutdown $(KVM_BUILD_DOMAIN)

From andrew.cagney at gmail.com  Wed Aug 21 12:43:35 2019
From: andrew.cagney at gmail.com (Andrew Cagney)
Date: Wed, 21 Aug 2019 08:43:35 -0400
Subject: [Swan-dev] kvm-install speed up run git commands on host
In-Reply-To: <20190820205411.guslygzlhubs6z7v@AntonyAntony.local>
References: <20190820205411.guslygzlhubs6z7v@AntonyAntony.local>
Message-ID: <CAJeAr6u6nTPnuW2kPDjf3hEf1_1+sejXKuLh78wyXiXBsc1yBg@mail.gmail.com>

On Tue, 20 Aug 2019 at 16:54, Antony Antony <antony at phenome.org> wrote:
>
> Here is an idea to speedup up building inside kvm when using "kvm-install"
> from what I red on IRC this would save upto 80seconds in the build.  Testrun
> could take 4hours.

It hurts when it matters and it also hurts the host (just not as bad)
- when trying to turn around a test result after (possibly) tweaking a
single .c file, vis:

   make base kvm-install kvm-test KVM_TESTS=testing/pluto/basic-pluto-01

Some time back I went through all this pain with the web code - Paul
pointed out it was crippling KVM.  From memory, the critical thing was
to drop -dirty (it's not exactly a must have) but I need to re-confirm
this.

>
> I am curious if it helps Andrew.
> I am trying avoid loosing features in the name saving few seconds.
>
> -antony
> _______________________________________________
> Swan-dev mailing list
> Swan-dev at lists.libreswan.org
> https://lists.libreswan.org/mailman/listinfo/swan-dev

From hugh at mimosa.com  Thu Aug 22 13:19:55 2019
From: hugh at mimosa.com (D. Hugh Redelmeier)
Date: Thu, 22 Aug 2019 09:19:55 -0400 (EDT)
Subject: [Swan-dev] man pages for our library routines
Message-ID: <alpine.LFD.2.21.1908220849100.20713@redeye.mimosa.com>

>From IRC:

<cagney>	DHR, unlike snprintf(), the *tot() return a length that 
includes the '\0'?

The way to answer this is to read the man pages.  Unfortunately, those 
have become unknown.

Henry created very comprehensive man pages for those library routines.

The FreeS/WAN and Openswan projects installed those pages so that, for 
example,
	man ipsec_addrtot
would display the page for addrtot.

The Libreswan project no longer builds or installs those pages.  But the 
source code for the pages still exists and can be consulted.  Albeit in 
XML rather than the original troff markup.

Since even some developers don't know of these pages' existence, there is 
a chance that the man pages are bitrotten.  But I have found few 
reasons to revise Henry's excellent code.  The major bitrot is probably 
that some routines have been deleted (perhaps without removing 
documentation) and some have been added (without providing documentation).

One consequence of a man page not being installed is that its aliases are 
not manifest.  For example, lib/libswan/ttoaddr.3.xml would be installed 
with additional links for ipsec_ttoaddr ipsec_tnatoaddr ipsec_addrtot ipsec_ttosubnet
ipsec_subnettot.

(I don't know how to view a manpage that is in XML.  The man command
would display one written in troff.  I'm old fashioned and don't
consider XML an improvement.  We're still fixing bug introduced by the
translation.)

Now for the answer to Andrew's original question.  Here's an extract
from lib/libswan/ttoaddr.3.xml

The binary-to-text functions return
<literal>0</literal>
for a failure, and otherwise
always return the size of buffer that would
be needed to
accommodate the full conversion result, including terminating NUL;
it is the caller's responsibility to check this against the size of
the provided buffer to determine whether truncation has occurred.</para>
</refsect1>

From andrew.cagney at gmail.com  Thu Aug 22 19:33:11 2019
From: andrew.cagney at gmail.com (Andrew Cagney)
Date: Thu, 22 Aug 2019 15:33:11 -0400
Subject: [Swan-dev] man pages for our library routines
In-Reply-To: <alpine.LFD.2.21.1908220849100.20713@redeye.mimosa.com>
References: <alpine.LFD.2.21.1908220849100.20713@redeye.mimosa.com>
Message-ID: <CAJeAr6tVUTaJFi3jFqxKb4rMDHhyBHMdatbu1-cUYp5oiej7cw@mail.gmail.com>

On Thu, 22 Aug 2019 at 09:20, D. Hugh Redelmeier <hugh at mimosa.com> wrote:
>
> From IRC:
>
> <cagney>        DHR, unlike snprintf(), the *tot() return a length that
> includes the '\0'?
>
> The way to answer this is to read the man pages.  Unfortunately, those
> have become unknown.
>
> Henry created very comprehensive man pages for those library routines.
>
> The FreeS/WAN and Openswan projects installed those pages so that, for
> example,
>         man ipsec_addrtot
> would display the page for addrtot.
>
> The Libreswan project no longer builds or installs those pages.  But the
> source code for the pages still exists and can be consulted.  Albeit in
> XML rather than the original troff markup.
>
> Since even some developers don't know of these pages' existence, there is
> a chance that the man pages are bitrotten.  But I have found few
> reasons to revise Henry's excellent code.  The major bitrot is probably
> that some routines have been deleted (perhaps without removing
> documentation) and some have been added (without providing documentation).
>
> One consequence of a man page not being installed is that its aliases are
> not manifest.  For example, lib/libswan/ttoaddr.3.xml would be installed
> with additional links for ipsec_ttoaddr ipsec_tnatoaddr ipsec_addrtot ipsec_ttosubnet
> ipsec_subnettot.

I suspect at some point there was also a library so installing the man
pages did made sense.  That's no longer the case.

They're neither read nor maintained because they are external to the
source code (and honestly reading the source is more reliable).

> (I don't know how to view a manpage that is in XML.  The man command
> would display one written in troff.  I'm old fashioned and don't
> consider XML an improvement.  We're still fixing bug introduced by the
> translation.)
>
> Now for the answer to Andrew's original question.  Here's an extract
> from lib/libswan/ttoaddr.3.xml
>
> The binary-to-text functions return
> <literal>0</literal>
> for a failure, and otherwise
> always return the size of buffer that would
> be needed to
> accommodate the full conversion result, including terminating NUL;
> it is the caller's responsibility to check this against the size of
> the provided buffer to determine whether truncation has occurred.</para>
> </refsect1>

Yes, I reverse engineered this from the code:

      len += *tot(..., buf+len, sizeof(buf) - len)
      buf[len] = ' ';
      len += *tot(....)

and my irc follow up was:

   cagney> DHR, really confusing

because the interface is not consistent with the *print*f() family -
trying to combine *printf() and *tot() isn't likely to end well.

From paul at nohats.ca  Thu Aug 22 20:46:01 2019
From: paul at nohats.ca (Paul Wouters)
Date: Thu, 22 Aug 2019 16:46:01 -0400
Subject: [Swan-dev] man pages for our library routines
In-Reply-To: <CAJeAr6tVUTaJFi3jFqxKb4rMDHhyBHMdatbu1-cUYp5oiej7cw@mail.gmail.com>
References: <alpine.LFD.2.21.1908220849100.20713@redeye.mimosa.com>
 <CAJeAr6tVUTaJFi3jFqxKb4rMDHhyBHMdatbu1-cUYp5oiej7cw@mail.gmail.com>
Message-ID: <9D5C361F-951D-4949-9065-142F8A11847F@nohats.ca>


> On Aug 22, 2019, at 15:33, Andrew Cagney <andrew.cagney at gmail.com> wrote:
> 
> 
> I suspect at some point there was also a library so installing the man
> pages did made sense.  That's no longer the case.

No, it was never a real library. It was basically freeswan developer information in the form of man pages. It was good but they were also already littered with ?this function is obsolete and will go away? which was a lie.

I stopped installing them because there were no real consumers for these and they were full of warnings from automatic lint tools etc and they were crappily converted from nroff to xml by me.

> 
> They're neither read nor maintained because they are external to the
> source code (and honestly reading the source is more reliable).

I agree

Paul


From andrew.cagney at gmail.com  Thu Aug 22 21:06:34 2019
From: andrew.cagney at gmail.com (Andrew Cagney)
Date: Thu, 22 Aug 2019 17:06:34 -0400
Subject: [Swan-dev] end.host_addr's port
Message-ID: <CAJeAr6uu8Xo8MB695t08mLvc4FZvrRo+QFtVojCS-TLb2tV3dQ@mail.gmail.com>

I'm looking at:

struct end {
       ip_address host_addr;
        ip_subnet client;
        bool has_client;
        bool has_client_wildcard;
        bool has_port_wildcard;
        uint16_t host_port;             /* where the IKE port is */
        uint16_t port;                  /* port number, if per-port keying */
}

and am puzzled by .port vs .host_port and .client vs .host[addr].  My
working theory was that things were paired:

   .client and .port (what IKEv2 calls a traffic selector)
   .host_addr and .host_port (the IKE endpoint)

but, in the case of .host_addr, the code seems to be fighting itself
over what the port should be.  For instance:

- in ikev2_ts.c the .host_addr's port is forced to the negotiated TS
client port:

    c->spd.that.client = tmp_subnet_r;
    c->spd.that.port = st->st_ts_that.startport;
    c->spd.that.protocol = st->st_ts_that.ipprotoid;
    setportof(htons(c->spd.that.port),
          &c->spd.that.host_addr);
    setportof(htons(c->spd.that.port),
          &c->spd.that.client.addr);

- but then in state.c:mobike, it's forced to the sender's port
(.sender has probably always had the port embedded in it).

        /* MOBIKE responder processing request */
        c->spd.that.host_addr = md->sender;
        c->spd.that.host_port = hportof(&md->sender);

A look at *_raw_eroute() shows .host_port is ignored (I thought it was
used, but it turns out that was only for prettying an error).

A look at .has_client shows more promise, the code seems to copy
.host_addr into .client vis:

    /* default client to subnet containing only self
     * XXX This may mean that the client's address family doesn't match
     * tunnel_addr_family.
     */
    if (!c->spd.that.has_client)
        addrtosubnet(&c->spd.that.host_addr, &c->spd.that.client);

and, I I'm guessing, is assuming that /host_addr's port is still set
to .port (the client port).

Andrew

From hugh at mimosa.com  Fri Aug 23 02:26:51 2019
From: hugh at mimosa.com (D. Hugh Redelmeier)
Date: Thu, 22 Aug 2019 22:26:51 -0400 (EDT)
Subject: [Swan-dev] man pages for our library routines
In-Reply-To: <9D5C361F-951D-4949-9065-142F8A11847F@nohats.ca>
References: <alpine.LFD.2.21.1908220849100.20713@redeye.mimosa.com>
 <CAJeAr6tVUTaJFi3jFqxKb4rMDHhyBHMdatbu1-cUYp5oiej7cw@mail.gmail.com>
 <9D5C361F-951D-4949-9065-142F8A11847F@nohats.ca>
Message-ID: <alpine.LFD.2.21.1908222150070.31568@redeye.mimosa.com>

| From: Paul Wouters <paul at nohats.ca>

| No, it was never a real library. It was basically freeswan developer 
| information in the form of man pages.

It was a library.  Interestingly, it did not depend on the rest of
FreeS/WAN.

It wasn't adopted by other projects, but that's too bad.  It created
some useful abstractions.  It was carefully designed and then evolved in 
light of experience.

One disaster seems to have been part of the Openswan project.  Lots of
other code was moved (badly) into the same tree as the original
library.  It didn't mix because it had various dependencies to code
outside the library.  So the original clean library interfaces were
muddied.  (I may have some of this wrong since I never have understood 
this new modularization.)

| It was good but they were also 
| already littered with ?this function is obsolete and will go away? which 
| was a lie.

Lies can and should be fixed.  Usually by changing the code, not the
documentation.  It would have been easy since all obsoleted functions
were replaced by more convenient ones (see "evolved", above).

(We're carrying around too much code that receives no maintenance.  If
we don't maintain it we should probably delete it.  I would guess that
only unmaintained code uses those obsolete functions.)

(Actually, worse than unmaintained code is untested code.  We dare not
maintain untested code for fear of breaking it without ever knowing.)

The claim that a function is obsolete isn't a lie.  That the obsolete
functions are still being used is not a failure of the library.  It is
a failure of the maintainers of the code that uses of those routines.

| I stopped installing them because there were no real consumers for these 
| and they were full of warnings from automatic lint tools etc and they 
| were crappily converted from nroff to xml by me.

I used them.  I was the main customer in FreeS/WAN.

I don't use them now because they aren't easy to use.  I miss them.

(I don't know (remember?) what "lint" you are referring to.  Perhaps I 
could have fixed it -- I used to know troff.)

| > On Aug 22, 2019, at 15:33, Andrew Cagney <andrew.cagney at gmail.com>
| wrote:

| > They're neither read nor maintained because they are external to the
| > source code (and honestly reading the source is more reliable).

No, they were used and maintained.  I used the man pages and not the
code to understand what the functions did.

Henry's coding was pretty reliable.  It did match the man page.

Example: your query was easily answered by reading the man page.

The code of those library routines has hardly been touched since Henry
wrote them.  That's because they were solid, simple, useful, and
stand-alone.

From antony at phenome.org  Fri Aug 23 12:27:27 2019
From: antony at phenome.org (Antony Antony)
Date: Fri, 23 Aug 2019 14:27:27 +0200
Subject: [Swan-dev] end.host_addr's port
In-Reply-To: <CAJeAr6uu8Xo8MB695t08mLvc4FZvrRo+QFtVojCS-TLb2tV3dQ@mail.gmail.com>
References: <CAJeAr6uu8Xo8MB695t08mLvc4FZvrRo+QFtVojCS-TLb2tV3dQ@mail.gmail.com>
Message-ID: <20190823122727.i7qjumntrx7j567t@AntonyAntony.local>

yes too many organically grown workarounds. MOBIKE and NAT stretched these 
ideas quite a bit.

I  am just sharing few  bits I re-collect from MOBIKE work.
btw have you noticed pluto_port yet? It is used as initial value host_port.

On Thu, Aug 22, 2019 at 05:06:34PM -0400, Andrew Cagney wrote:
> I'm looking at:
> 
> struct end {
>        ip_address host_addr;
>         ip_subnet client;
>         bool has_client;
>         bool has_client_wildcard;
>         bool has_port_wildcard;
>         uint16_t host_port;             /* where the IKE port is */
>         uint16_t port;                  /* port number, if per-port keying */
> }
> 
> and am puzzled by .port vs .host_port and .client vs .host[addr].  My
> working theory was that things were paired:
> 
>    .client and .port (what IKEv2 calls a traffic selector)
>    .host_addr and .host_port (the IKE endpoint)

This theory sounds good, possibly over simplistic. It is missing the part 
dynamic change. The need copy from c to st and vice versa is complicated with 
MOBIKE. It touch what is in c and what is in st, especially for an 
instantiated connection.

for the record IKE is an IKEv2 termonlogy. during IKEv1 only days isakmp was 
popular.
 
> but, in the case of .host_addr, the code seems to be fighting itself
> over what the port should be.  For instance:
> 
> - in ikev2_ts.c the .host_addr's port is forced to the negotiated TS
> client port:
> 
>     c->spd.that.client = tmp_subnet_r;
>     c->spd.that.port = st->st_ts_that.startport;
>     c->spd.that.protocol = st->st_ts_that.ipprotoid;
>     setportof(htons(c->spd.that.port),
>           &c->spd.that.host_addr);
>     setportof(htons(c->spd.that.port),
>           &c->spd.that.client.addr);
> 
> - but then in state.c:mobike, it's forced to the sender's port
> (.sender has probably always had the port embedded in it).
> 
>         /* MOBIKE responder processing request */
>         c->spd.that.host_addr = md->sender;
>         c->spd.that.host_port = hportof(&md->sender);

My recollections are vague and I didn't double checked it in git master.

you need IKE port in a connection for two reasons. To survive a restart, 
short interruptions, say dpd. Second for output "ipsec status"?

Note when initiating a connection a call to set_state_ike_endpoints.

 set_state_ike_endpoints(st, c)
	st->st_remoteaddr = c->spd.that.host_addr;
	st->st_remoteport = c->spd.that.host_port;

While adding MOBIKE, Paul and I realized copying port and host_addr to c 
would be a limitation for a longer connection interruptions and restart.
However, we used a reasoning most use cases are only for right=%any. Those 
connections wouldn't initiate after say a dpd restart or when the other end
reboots.

The MOBIKE overwriting host_port and host_addr in c is a limitation!
In most use cases Libreswan is MOBIKE responder with right=%any so it sort 
of works. That is why you are seeing overwriting what is in c, which is 
confusing.

on the responder one odd bit for me is:
calls in  instantiate()
	port = htons(dst->port);
	setportof(port, &dst->host_addr);

> A look at *_raw_eroute() shows .host_port is ignored (I thought it was
> used, but it turns out that was only for prettying an error).
> 
> A look at .has_client shows more promise, the code seems to copy
> .host_addr into .client vis:
> 
>     /* default client to subnet containing only self
>      * XXX This may mean that the client's address family doesn't match
>      * tunnel_addr_family.
>      */
>     if (!c->spd.that.has_client)
>         addrtosubnet(&c->spd.that.host_addr, &c->spd.that.client);
> 
> and, I I'm guessing, is assuming that /host_addr's port is still set
> to .port (the client port).

Also note the comparisons to pluto_port. 

I think we can delete some of them.
c->spd.this.port can go, use c->spd.this.client.addr.port instead

host_port can be ditched in favor of host_addr's port. This might be a bit 
invasive change in v1 and v2.

I wonder what to do with ikev2 ts, same information has multiple copies in 
st, and in c.

while we are at it, recent changes bc4d25c065b  #ifdef ENDPOINT_ADDRESS_PORT 
is a bit confusing to me. These started a while ago? 25bcb7f6cb0. Too many 
changes in flight.

From paul at nohats.ca  Fri Aug 23 14:41:02 2019
From: paul at nohats.ca (Paul Wouters)
Date: Fri, 23 Aug 2019 10:41:02 -0400 (EDT)
Subject: [Swan-dev] end.host_addr's port
In-Reply-To: <20190823122727.i7qjumntrx7j567t@AntonyAntony.local>
References: <CAJeAr6uu8Xo8MB695t08mLvc4FZvrRo+QFtVojCS-TLb2tV3dQ@mail.gmail.com>
 <20190823122727.i7qjumntrx7j567t@AntonyAntony.local>
Message-ID: <alpine.LRH.2.21.1908231019420.30045@bofh.nohats.ca>

On Fri, 23 Aug 2019, Antony Antony wrote:

> yes too many organically grown workarounds. MOBIKE and NAT stretched these
> ideas quite a bit.

Indeed.

>> struct end {
>>        ip_address host_addr;
>>         ip_subnet client;
>>         bool has_client;
>>         bool has_client_wildcard;
>>         bool has_port_wildcard;
>>         uint16_t host_port;             /* where the IKE port is */
>>         uint16_t port;                  /* port number, if per-port keying */
>> }
>>
>> and am puzzled by .port vs .host_port and .client vs .host[addr].  My
>> working theory was that things were paired:

Some of the issues stem from freeswan's original concept of host with or
without subnets. It copies from host* to client* if no subnets are set.

I think Hugh used the per port keying to run multiple plutos on the same
host for testing decades ago. I'm not sure if that's where the "port" vs
"host_port" came from.

It is also possible that it came in with the NAT-T stuff, where we went
from IKE port to "IKE port or NAT IKE port".

It would make sense to clean this up. For instance whether or not to use
the NAT IKE port (4500) mostly depends on whether NAT was detected, so
that property should live in the state and not in c->spd.this/that.

>>    .client and .port (what IKEv2 calls a traffic selector)
>>    .host_addr and .host_port (the IKE endpoint)
>
> This theory sounds good, possibly over simplistic. It is missing the part
> dynamic change. The need copy from c to st and vice versa is complicated with
> MOBIKE. It touch what is in c and what is in st, especially for an
> instantiated connection.

Yes, I avoided the understanding and cleanup you are doing now when I
worked on MOBIKE. I tried to keep everything in struct state that would
"override" things, and tried to not change struct connection as much as
possible.

>> but, in the case of .host_addr, the code seems to be fighting itself
>> over what the port should be.  For instance:
>>
>> - in ikev2_ts.c the .host_addr's port is forced to the negotiated TS
>> client port:
>>
>>     c->spd.that.client = tmp_subnet_r;
>>     c->spd.that.port = st->st_ts_that.startport;
>>     c->spd.that.protocol = st->st_ts_that.ipprotoid;
>>     setportof(htons(c->spd.that.port),
>>           &c->spd.that.host_addr);
>>     setportof(htons(c->spd.that.port),
>>           &c->spd.that.client.addr);

The traffic selectors and the IKE address and port _should_ not be
related or mixed up or sync'ed in theory. But with "narrowing" we
have the case where the traffic selector set might be a subset of
the c->spd.that/this.client and c->spd.this/that.[port|protocol].
So what we then do is instantiate the connection, and only once
scrible over the this/that properties. From then on, these are
again "immutable" for that (instantiated) connection's lifetime.
Note that a connection supporting narrowing is for that reason
created as a CK_TEMPLATE, so even if the TS covers everything,
it is instantiated.

>> - but then in state.c:mobike, it's forced to the sender's port
>> (.sender has probably always had the port embedded in it).
>>
>>         /* MOBIKE responder processing request */
>>         c->spd.that.host_addr = md->sender;
>>         c->spd.that.host_port = hportof(&md->sender);
>
> My recollections are vague and I didn't double checked it in git master.
>
> you need IKE port in a connection for two reasons. To survive a restart,
> short interruptions, say dpd. Second for output "ipsec status"?

I think the copying from sender is more to survive NAT changes. If the
NAT router maps the port differently, we need to change our end to send
to this new (IP/)port. But we should only do that after we decrypted and
authenticated the packet as "new" so a forged replay would not force us
back. This is the area where there is some complicated interaction
between NAT routers changing things and MOBIKE changing things.

> Note when initiating a connection a call to set_state_ike_endpoints.
>
> set_state_ike_endpoints(st, c)
> 	st->st_remoteaddr = c->spd.that.host_addr;
> 	st->st_remoteport = c->spd.that.host_port;
>
> While adding MOBIKE, Paul and I realized copying port and host_addr to c
> would be a limitation for a longer connection interruptions and restart.
> However, we used a reasoning most use cases are only for right=%any. Those
> connections wouldn't initiate after say a dpd restart or when the other end
> reboots.
>
> The MOBIKE overwriting host_port and host_addr in c is a limitation!
> In most use cases Libreswan is MOBIKE responder with right=%any so it sort
> of works. That is why you are seeing overwriting what is in c, which is
> confusing.

Right. Although in this case, if the information is struct state was
more reliable, we could potentially not override the connection. Note
MOBIKE is incompatible with transport mode so there is always a
this/that client when we are changing the host addr/port.

> on the responder one odd bit for me is:
> calls in  instantiate()
> 	port = htons(dst->port);
> 	setportof(port, &dst->host_addr);

I have no memory of this specifically. It could be that this originally
comes from the md->st where the ip/port info lives before we have
matched a connection. Then when we match it, we might have (due to hairy
code) overwritten the connection's port to match the sender port in the
md.

>> A look at *_raw_eroute() shows .host_port is ignored (I thought it was
>> used, but it turns out that was only for prettying an error).

Yes, eroutes should only work on *client, not *host. And for transport
mode and for host-to-host tunnel mode connections, the host variables
are copied as client variables.

>> A look at .has_client shows more promise, the code seems to copy
>> .host_addr into .client vis:
>>
>>     /* default client to subnet containing only self
>>      * XXX This may mean that the client's address family doesn't match
>>      * tunnel_addr_family.
>>      */
>>     if (!c->spd.that.has_client)
>>         addrtosubnet(&c->spd.that.host_addr, &c->spd.that.client);
>>
>> and, I I'm guessing, is assuming that /host_addr's port is still set
>> to .port (the client port).
>
> Also note the comparisons to pluto_port.
>
> I think we can delete some of them.
> c->spd.this.port can go, use c->spd.this.client.addr.port instead

I think that is the wrong abstraction. The this/that.client should only
relate to the ipsec policy for src/dst and not contain anything related
to the host IP or port used for IKE. So in my opinion,
c->spd.this.client.addr.port should be removed or always be 0.

> host_port can be ditched in favor of host_addr's port. This might be a bit
> invasive change in v1 and v2.

Additionally, I'm about to add code to support leftikeport= and
rightikeport= to support openshift/kubernetes containers. So if we are
doing a cleanup, which is much needed, let's ensure we then also take
this new feature into account.

> I wonder what to do with ikev2 ts, same information has multiple copies in
> st, and in c.

Perhaps rewrite this only after we have ditched IKEv1 ?

> while we are at it, recent changes bc4d25c065b  #ifdef ENDPOINT_ADDRESS_PORT
> is a bit confusing to me. These started a while ago? 25bcb7f6cb0. Too many
> changes in flight.

I think this is Andrew's way of porting code and testing before flipping
everything to "new method" when he removes the define ?

Paul

From andrew.cagney at gmail.com  Sat Aug 24 18:35:41 2019
From: andrew.cagney at gmail.com (Andrew Cagney)
Date: Sat, 24 Aug 2019 14:35:41 -0400
Subject: [Swan-dev] end.host_addr's port
In-Reply-To: <alpine.LRH.2.21.1908231019420.30045@bofh.nohats.ca>
References: <CAJeAr6uu8Xo8MB695t08mLvc4FZvrRo+QFtVojCS-TLb2tV3dQ@mail.gmail.com>
 <20190823122727.i7qjumntrx7j567t@AntonyAntony.local>
 <alpine.LRH.2.21.1908231019420.30045@bofh.nohats.ca>
Message-ID: <CAJeAr6ugov5cFNRtn3b+iw+uq0O2b=TLEFG7U7VQQtRvbo=dJw@mail.gmail.com>

I thought I had one problem; now I've got two!


On Fri, 23 Aug 2019 at 10:41, Paul Wouters <paul at nohats.ca> wrote:
>
> On Fri, 23 Aug 2019, Antony Antony wrote:
>
> > yes too many organically grown workarounds. MOBIKE and NAT stretched these
> > ideas quite a bit.
>
> Indeed.
>
> >> struct end {
> >>        ip_address host_addr;
> >>         ip_subnet client;
> >>         bool has_client;
> >>         bool has_client_wildcard;
> >>         bool has_port_wildcard;
> >>         uint16_t host_port;             /* where the IKE port is */
> >>         uint16_t port;                  /* port number, if per-port keying */
> >> }
> >>
> >> and am puzzled by .port vs .host_port and .client vs .host[addr].  My
> >> working theory was that things were paired:
>
> Some of the issues stem from freeswan's original concept of host with or
> without subnets. It copies from host* to client* if no subnets are set.
>
> I think Hugh used the per port keying to run multiple plutos on the same
> host for testing decades ago. I'm not sure if that's where the "port" vs
> "host_port" came from.

That might explain:
        bool host_port_specific;        /* if TRUE, then IKE ports are
tested for */
which seems to only change a log line

> It is also possible that it came in with the NAT-T stuff, where we went
> from IKE port to "IKE port or NAT IKE port".
>
> It would make sense to clean this up. For instance whether or not to use
> the NAT IKE port (4500) mostly depends on whether NAT was detected, so
> that property should live in the state and not in c->spd.this/that.
>
> >>    .client and .port (what IKEv2 calls a traffic selector)
> >>    .host_addr and .host_port (the IKE endpoint)
> >
> > This theory sounds good, possibly over simplistic. It is missing the part
> > dynamic change. The need copy from c to st and vice versa is complicated with
> > MOBIKE. It touch what is in c and what is in st, especially for an
> > instantiated connection.

The dynamic change should only conceptually affect .host_addr / .host_port.
(but might also cause .client to change if there isn't a client).

> Yes, I avoided the understanding and cleanup you are doing now when I
> worked on MOBIKE. I tried to keep everything in struct state that would
> "override" things, and tried to not change struct connection as much as
> possible.
>
> >> but, in the case of .host_addr, the code seems to be fighting itself
> >> over what the port should be.  For instance:
> >>
> >> - in ikev2_ts.c the .host_addr's port is forced to the negotiated TS
> >> client port:
> >>
> >>     c->spd.that.client = tmp_subnet_r;
> >>     c->spd.that.port = st->st_ts_that.startport;
> >>     c->spd.that.protocol = st->st_ts_that.ipprotoid;
> >>     setportof(htons(c->spd.that.port),
> >>           &c->spd.that.host_addr);
> >>     setportof(htons(c->spd.that.port),
> >>           &c->spd.that.client.addr);
>
> The traffic selectors and the IKE address and port _should_ not be
> related or mixed up or sync'ed in theory. But with "narrowing" we
> have the case where the traffic selector set might be a subset of
> the c->spd.that/this.client and c->spd.this/that.[port|protocol].
> So what we then do is instantiate the connection, and only once
> scrible over the this/that properties. From then on, these are
> again "immutable" for that (instantiated) connection's lifetime.
> Note that a connection supporting narrowing is for that reason
> created as a CK_TEMPLATE, so even if the TS covers everything,
> it is instantiated.
>
> >> - but then in state.c:mobike, it's forced to the sender's port
> >> (.sender has probably always had the port embedded in it).
> >>
> >>         /* MOBIKE responder processing request */
> >>         c->spd.that.host_addr = md->sender;
> >>         c->spd.that.host_port = hportof(&md->sender);
> >
> > My recollections are vague and I didn't double checked it in git master.
> >
> > you need IKE port in a connection for two reasons. To survive a restart,
> > short interruptions, say dpd. Second for output "ipsec status"?
>
> I think the copying from sender is more to survive NAT changes. If the
> NAT router maps the port differently, we need to change our end to send
> to this new (IP/)port. But we should only do that after we decrypted and
> authenticated the packet as "new" so a forged replay would not force us
> back. This is the area where there is some complicated interaction
> between NAT routers changing things and MOBIKE changing things.
>
> > Note when initiating a connection a call to set_state_ike_endpoints.
> >
> > set_state_ike_endpoints(st, c)
> >       st->st_remoteaddr = c->spd.that.host_addr;
> >       st->st_remoteport = c->spd.that.host_port;
> >
> > While adding MOBIKE, Paul and I realized copying port and host_addr to c
> > would be a limitation for a longer connection interruptions and restart.
> > However, we used a reasoning most use cases are only for right=%any. Those
> > connections wouldn't initiate after say a dpd restart or when the other end
> > reboots.
> >
> > The MOBIKE overwriting host_port and host_addr in c is a limitation!
> > In most use cases Libreswan is MOBIKE responder with right=%any so it sort
> > of works. That is why you are seeing overwriting what is in c, which is
> > confusing.
>
> Right. Although in this case, if the information is struct state was
> more reliable, we could potentially not override the connection. Note
> MOBIKE is incompatible with transport mode so there is always a
> this/that client when we are changing the host addr/port.
>
> > on the responder one odd bit for me is:
> > calls in  instantiate()
> >       port = htons(dst->port);
> >       setportof(port, &dst->host_addr);
>
> I have no memory of this specifically. It could be that this originally
> comes from the md->st where the ip/port info lives before we have
> matched a connection. Then when we match it, we might have (due to hairy
> code) overwritten the connection's port to match the sender port in the
> md.
>
> >> A look at *_raw_eroute() shows .host_port is ignored (I thought it was
> >> used, but it turns out that was only for prettying an error).
>
> Yes, eroutes should only work on *client, not *host. And for transport
> mode and for host-to-host tunnel mode connections, the host variables
> are copied as client variables.
>
> >> A look at .has_client shows more promise, the code seems to copy
> >> .host_addr into .client vis:
> >>
> >>     /* default client to subnet containing only self
> >>      * XXX This may mean that the client's address family doesn't match
> >>      * tunnel_addr_family.
> >>      */
> >>     if (!c->spd.that.has_client)
> >>         addrtosubnet(&c->spd.that.host_addr, &c->spd.that.client);
> >>
> >> and, I I'm guessing, is assuming that /host_addr's port is still set
> >> to .port (the client port).
> >
> > Also note the comparisons to pluto_port.
> >
> > I think we can delete some of them.
> > c->spd.this.port can go, use c->spd.this.client.addr.port instead
>
> I think that is the wrong abstraction. The this/that.client should only
> relate to the ipsec policy for src/dst and not contain anything related
> to the host IP or port used for IKE. So in my opinion,
> c->spd.this.client.addr.port should be removed or always be 0.
>
> > host_port can be ditched in favor of host_addr's port. This might be a bit
> > invasive change in v1 and v2.
>
> Additionally, I'm about to add code to support leftikeport= and
> rightikeport= to support openshift/kubernetes containers. So if we are
> doing a cleanup, which is much needed, let's ensure we then also take
> this new feature into account.
>
> > I wonder what to do with ikev2 ts, same information has multiple copies in
> > st, and in c.
>
> Perhaps rewrite this only after we have ditched IKEv1 ?
>
> > while we are at it, recent changes bc4d25c065b  #ifdef ENDPOINT_ADDRESS_PORT
> > is a bit confusing to me. These started a while ago? 25bcb7f6cb0. Too many
> > changes in flight.
>
> I think this is Andrew's way of porting code and testing before flipping
> everything to "new method" when he removes the define ?

Two things:

- it wraps the current working theory on what the structure should
look like in a #ifdef
Of course the theory evolves, for instance I suspect adding
uint8_t[16] to ip_address so that static initialisation is possible
using hardwired bytes

- when defined it breaks the compile in a way that helps flush out
weird and wonderful problems, like the above, which is exposed because
code is trying to pass a simple address to functions expecting
address+port

From andrew.cagney at gmail.com  Fri Aug 30 22:02:28 2019
From: andrew.cagney at gmail.com (Andrew Cagney)
Date: Fri, 30 Aug 2019 18:02:28 -0400
Subject: [Swan-dev] [Swan-commit] Changes to ref refs/heads/master
In-Reply-To: <46K1JK4MTYz7QQLX@vault.libreswan.fi>
References: <46K1JK4MTYz7QQLX@vault.libreswan.fi>
Message-ID: <CAJeAr6vUNKzzWWka3r9rUcaHBym3EmwTFMnW4wE2OznW0+k8rQ@mail.gmail.com>

See https://lists.libreswan.org/pipermail/swan-dev/2019-August/003358.html

On Thu, 29 Aug 2019 at 07:55, Antony Antony <antony at vault.libreswan.fi> wrote:
>
> New commits:
> commit aff9299805c53ba88bc3c8217185080db86c4610
> Author: Antony Antony <antony at phenome.org>
> Date:   Thu Aug 29 11:52:35 2019 +0000
>
>     Revert "libswan: re-create version.c every time run a make"
>
>     not good to merge to master. It will trigger on "make install-base"
>
>     This reverts commit 3bfd6da587780f0cd06131e2c41ffa9a47d2a70d.