<div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div>Followed all your suggestions and the connection information shows the that the oppo sees that IP addresses across the connection down to the %fromcert. What's different this time is the +MS+S=C which I have no idea what that is. I blew away the /etc/ipsec.d/*.db and when back to the instruction on how to create it.<br><br>oppo_instantiate() instantiated "[1] ...XX.XXX.XXX.205"private#<a href="http://0.0.0.0/0">0.0.0.0/0</a>: 10.0.0.47[MS+S=C]---13.57.200.87...XX.XXX.XXX.205[%fromcert,+MC+S=C]<br></div><div><br></div><div>Oct 7 18:54:28.198237: | private key for cert Thor not found in local cache; loading from NSS DB<br>Oct 7 18:54:28.202605: | ikev2_child_sa_respond returned STF_FAIL+v2N_TS_UNACCEPTABLE<br>Oct 7 18:54:28.202620: | ikev2_parent_inI2outR2_continue_tail returned STF_FAIL+v2N_TS_UNACCEPTABLE<br>Oct 7 18:54:28.202628: "private#<a href="http://0.0.0.0/0">0.0.0.0/0</a>"[1] ...XX.XXX.XXX.205 #1: responding to AUTH message (ID 1) from XX.XXX.XXX.205:47402 with encrypted notification AUTHENTICATION_FAILED<br></div><div>-------------------------<br></div><div><br></div><div>root@thor:/etc/ipsec.d# cat nsspassword <br>NSS Certificate DB:12345678<br>-------------------------<br>root@thor:/etc/ipsec.d# certutil -d sql:/etc/ipsec.d/ -L <br><br>Certificate Nickname Trust Attributes<br> SSL,S/MIME,JAR/XPI<br>Thor u,u,u<br>5 Dev CA CT,, <br>VPN Subordinate CT,, <br></div><div>-------------------------</div><div><br></div><div>root@thor:/etc/ipsec.d# certutil -d sql:/etc/ipsec.d/ -K<br>certutil: Checking token "NSS Certificate DB" in slot "NSS User Private Key and Certificate Services"<br>Enter Password or Pin for "NSS Certificate DB":<br>< 0> rsa e392b3b8e90f3629e6fefa741f2c89db4a5935e0 Thor<br></div><div><br></div><div>-------------------------</div><div>conn private<br> # IPsec mandatory<br> hostaddrfamily=ipv4<br> rightrsasigkey=%cert<br> right=%opportunisticgroup<br> rightid=%fromcert<br> rightca=%same<br> rightaddresspool=100.64.1.0-100.64.1.254<br> #rightcat=yes<br> #rightmodecfgclient=yes<br> #leftsubnet=<a href="http://10.0.0.47/32">10.0.0.47/32</a><br> left=%defaultroute<br> leftcert=Thor<br> leftsendcert=always<br> leftid=%cert<br> leftnexthop=13.57.200.87<br> #leftrsasigkey=%cert<br> narrowing=yes<br> type=tunnel<br> ikev2=insist<br> negotiationshunt=hold<br> failureshunt=drop<br> keyingtries=0<br> retransmit-timeout=3s<br> auto=ondemand<br> ike=aes256-sha1;modp2048<br> phase2alg=aes256-sha1;modp2048<br></div><div>-------------------------</div><div>root@thor:/etc/ipsec.d# ipsec auto --listcerts<br>000 <br>000 List of X.509 End Certificates:<br>000 <br>000 End certificate "Thor" - SN: 0x73f1b651ab0f019d<br>000 subject: C=US, ST=CA, L=Palo Alto, O=mycompany, OU=5, CN=Thor, E=<a href="mailto:admin@mycompany.com">admin@mycompany.com</a><br>000 issuer: CN= VPN Subordinate<br>000 not before: Fri Sep 28 05:50:47 2018<br>000 not after: Sun Sep 27 05:50:47 2020<br>000 2048 bit RSA: has private key<br>root@thor:/etc/ipsec.d# ipsec auto --listall<br>000 <br>000 List of Public Keys:<br>000 <br>000 Oct 07 18:54:28 2018, 2048 RSA Key AwEAAbcJJ (no private key), until Aug 25 18:23:07 2020 ok<br>000 ID_DER_ASN1_DN 'C=US, ST=CA, L=Palo Alto, O=mycompany, OU=, CN=ipsechost1, E=<a href="mailto:admin@mycompany.com">admin@mycompany.com</a>'<br>000 Issuer 'CN= VPN Subordinate'<br>000 Oct 07 18:54:28 2018, 2048 RSA Key AwEAAbcJJ (no private key), until Aug 25 18:23:07 2020 ok<br>000 ID_USER_FQDN '<a href="mailto:admin@mycompany.com">admin@mycompany.com</a>'<br>000 Issuer 'CN= VPN Subordinate'<br>000 Oct 07 18:54:20 2018, 2048 RSA Key AwEAAeaaN (has private key), until Sep 27 05:50:47 2020 ok<br>000 ID_DER_ASN1_DN 'C=US, ST=CA, L=Palo Alto, O=mycompany, OU=5, CN=Thor, E=<a href="mailto:admin@mycompany.com">admin@mycompany.com</a>'<br>000 Issuer 'CN= VPN Subordinate'<br>000 <br>000 List of Pre-shared secrets (from /etc/ipsec.secrets)<br>000 <br>000 0: RSA (none) (none)<br>000 <br>000 List of X.509 End Certificates:<br>000 <br>000 End certificate "Thor" - SN: 0x73f1b651ab0f019d<br>000 subject: C=US, ST=CA, L=Palo Alto, O=mycompany, OU=5, CN=Thor, E=<a href="mailto:admin@mycompany.com">admin@mycompany.com</a><br>000 issuer: CN= VPN Subordinate<br>000 not before: Fri Sep 28 05:50:47 2018<br>000 not after: Sun Sep 27 05:50:47 2020<br>000 2048 bit RSA: has private key<br>000 <br>000 List of X.509 CA Certificates:<br>000 <br>000 CA certificate " VPN Subordinate" - SN: 0x18f0fd4df52ebd9c<br>000 subject: CN= VPN Subordinate<br>000 issuer: CN= Dev CA<br>000 not before: Thu Aug 02 02:00:42 2018<br>000 not after: Mon May 05 00:00:00 2025<br>000 4096 bit RSA<br>000 <br>000 Root CA certificate " Dev CA" - SN: 0x09f888ec6ccbdc39<br>000 subject: CN= Dev CA<br>000 issuer: CN= Dev CA<br>000 not before: Tue Jun 26 04:30:58 2018<br>000 not after: Tue Jun 25 00:00:00 2030<br>000 2048 bit RSA<br>000 <br>000 List of CRLs:<br><br></div></div></div></div></div></div></div></div></div><br><div class="gmail_quote"><div dir="ltr">On Sun, Oct 7, 2018 at 10:25 AM rayv33n <<a href="mailto:rayv33n@gmail.com">rayv33n@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div>HI Paul,<br><br></div>Thanks for the response! I've been doing what you said users would be doing in your video on OE which is copying configs off the internet and trying different unrelated things. :D I'm guilty!<br><br></div>The regular config I have work if there is not NAT involved. Same config applies to all hosts in the lab and they work beautifully. Then I built an AWS instanced and tried it. Same Ubuntu systems, same everything but the only difference is AWS NAT and yes the lab systems are NAT as well.<br><br></div>This is what led to my confusion about the cause. I'll have a look at the pkcs12 import process and report back, it's possible that I screwed that up but in general I've been using the CN=hostname as the friendly name on import so it make it easier to remember. I also wanted to build the worse case scenario so when I showcased the demo I could start driving the narrative away from traditional firewalls/NAT/v4 and move it towards no-NAT and v6. That value of moving away from NAT and v4 is it's less complex to manage assuming v6 will do OE because the configuration can be a repeatable standard with no manually edits.<br><br></div>Let me do some checking and get back to you on the rest below.<br><div><div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr">On Fri, Oct 5, 2018 at 11:49 AM Paul Wouters <<a href="mailto:paul@nohats.ca" target="_blank">paul@nohats.ca</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Thu, 4 Oct 2018, rayv33n wrote:<br>
<br>
> No sure that to make of this message. Originally I thought it was a warning letting me know that after restarting ipsec that cache was check<br>
> and then private key<br>
<br>
The leftcert= entry has to match the "friendly name" that normally comes<br>
from importing the PKCS#12. I assume it can also be set when importing<br>
the signed certificate back into the NSS db.<br>
<br>
> I'm fairly certain this is a NAT issue but tcpdump<br>
> show 4500/UDP being used immediately after initial handshake.<br>
<br>
The cert errors have nothing to do with NAT. The NAT will only cause<br>
changes in the final IP addreses used for the src-dst of the IPsec<br>
tunnel. It does not affect authentication.<br>
<br>
<br>
> where to look After days of<br>
> unsuccessfully troubleshoot I'm finally coming to the list. Goals it so make a mesh network hosts only and not servers across lots of divers<br>
> infrastructure.<br>
> <br>
> network setup. ipsechost1(172.16.1.61)---> Netgate(76.1XX.2XX.2XX.2xx) <--> AWS(13.57.XXX.XX) --> Thor(10.0.0.47)<br>
<br>
So both ends are behind NAT? That gets a lot trickier, because both ends<br>
cannot request an address and offer one from a pool. Or is one of these<br>
perhaps a static port forward?<br>
<br>
> Using NSS of course with /etc/ipsec.d/nsspassword(NSS Certificate DB:12345678)<br>
<br>
so the configuration differs whether or not you need to support NAT, and<br>
whether to support is as the client behind NAT or the server (not NATed<br>
or perhaps a static port forward)<br>
<br>
> #Thor god of thunder<br>
> conn private<br>
> # IPsec mandatory<br>
> hostaddrfamily=ipv4<br>
> rightrsasigkey=%cert<br>
> right=%opportunisticgroup<br>
> rightid=%fromcert<br>
> rightca=%same<br>
> rightmodecfgclient=yes<br>
> leftsubnet=<a href="http://10.0.0.47/32" rel="noreferrer" target="_blank">10.0.0.47/32</a><br>
<br>
You should not specify any subnet= because the Opportunistic mechanism<br>
fills that in when instantiating the connection. This conn private is<br>
a template, so it can never have all the right addresses pre-filled in.<br>
<br>
> left=%defaultroute<br>
> leftcert=Thor<br>
> leftsendcert=always<br>
> leftid=%cert<br>
<br>
That is %fromcert, not %cert<br>
<br>
> leftnexthop=13.57.xxx.xx<br>
<br>
don't use any nexthop's.<br>
<br>
> leftrsasigkey=%cert<br>
> #narrowing=yes<br>
<br>
You need narrowing=yes if you can be behind NAT. It will allow the<br>
server to narrow you down. For the server side, you then need to<br>
specify a rightaddresspool= (used for the "NAT within IPsec", so you<br>
can pick eg <a href="http://100.64.0.0/16" rel="noreferrer" target="_blank">100.64.0.0/16</a>)<br>
<br>
> type=tunnel<br>
> ikev2=insist<br>
> negotiationshunt=hold<br>
> failureshunt=drop<br>
> keyingtries=0<br>
> retransmit-timeout=3s<br>
> auto=ondemand<br>
> ike=aes256-sha1;modp2048<br>
> phase2alg=aes256-sha1;modp2048<br>
<br>
will libreswan everywhere, it is better to leave out ike= esp= and<br>
phase2alg= lines and rely on the buildin defaults. It will make<br>
upgrades in the future easier.<br>
<br>
> Oct 4 14:38:50.506799: "private#<a href="http://0.0.0.0/0" rel="noreferrer" target="_blank">0.0.0.0/0</a>"[1] ...76.1XX.2XX.2XX.2xx #1: certificate verified OK:<br>
> E=<a href="mailto:admin@mycompany.com" target="_blank">admin@mycompany.com</a>,CN=ipsechost1,OU=Level5,O=mycompany,L=Palo Alto,ST=CA,C=US<br>
> Oct 4 14:38:50.507848: "private#<a href="http://0.0.0.0/0" rel="noreferrer" target="_blank">0.0.0.0/0</a>"[1] ...76.1XX.2XX.2XX.2xx #1: Authenticated using RSA<br>
> private key for cert Thor not found in local cache; loading from NSS DB<br>
> searching for certificate PKK_RSA:AwEAAeaaN vs PKK_RSA:AwEAAeaaN<br>
<br>
It found the one it wanted but no private key. This problem is unrelated<br>
to the Opportunistic configuration. You would have the same with a<br>
static vpn tunnel.<br>
<br>
Paul<br>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr" class="m_-265617599664004504gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">You are FREE to become a slave<br><div><br></div><div>Key ID: <span style="color:rgb(46,46,46);font-family:'Helvetica Neue',Helvetica,Arial,sans-serif;font-style:italic">9A452ABAA4593489</span></div><div><span style="color:rgb(46,46,46);font-family:'Helvetica Neue',Helvetica,Arial,sans-serif;font-style:italic">Finger Print: </span><span style="color:rgb(46,46,46);font-family:'Helvetica Neue',Helvetica,Arial,sans-serif;font-style:italic">7A8A 5849 ED44 52B1 0D8A EDAC 9A45 2ABA A459 3489</span></div><div><font color="#2e2e2e" face="Helvetica Neue, Helvetica, Arial, sans-serif"><i>Pub Key: </i></font><a href="http://pgp.mit.edu:11371/pks/lookup?search=rayv33n%40gmail.com&op=index" target="_blank">http://pgp.mit.edu:11371/pks/lookup?search=rayv33n%40gmail.com&op=index</a></div></div></div>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">You are FREE to become a slave<br><div><br></div><div>Key ID: <span style="color:rgb(46,46,46);font-family:'Helvetica Neue',Helvetica,Arial,sans-serif;font-style:italic">9A452ABAA4593489</span></div><div><span style="color:rgb(46,46,46);font-family:'Helvetica Neue',Helvetica,Arial,sans-serif;font-style:italic">Finger Print: </span><span style="color:rgb(46,46,46);font-family:'Helvetica Neue',Helvetica,Arial,sans-serif;font-style:italic">7A8A 5849 ED44 52B1 0D8A EDAC 9A45 2ABA A459 3489</span></div><div><font color="#2e2e2e" face="Helvetica Neue, Helvetica, Arial, sans-serif"><i>Pub Key: </i></font><a href="http://pgp.mit.edu:11371/pks/lookup?search=rayv33n%40gmail.com&op=index" target="_blank">http://pgp.mit.edu:11371/pks/lookup?search=rayv33n%40gmail.com&op=index</a></div></div></div>