[Swan] RSA key length or ID matching issue in 3.25 FIPS mode

[Matt Hilt]

I'm running opportunistic encryption between a number of servers all using RHEL 7 with FIPS mode enabled. Everything has been working fine using some RSA keys and libreswan 3.23.  RHEL now has 3.25 available, but when upgrading it warned that the RSA bit length was required to be > 3072.

No problem - I switched to some 4096 bit RSA keys and again everything worked fine on 3.23. However, 3.25 is again complaining. The errors come in two forms:

3.23 <--> 3.25
  - The 3.25 system still gives the following errors:
        FIPS: Rejecting cert with key size under 3072
        no RSA public key known for '%fromcert'
        Digital Signature authentication failed

3.25 <--> 3.25
- The initiating host has the following errors:
        DigSig: failed to find our RSA key


When running `ipsec whack --listall` I clearly see that the certs and each of its trust chain CAs are all 4096 bit RSA. Each server also reports "has private key" for their own cert.

I was using leftid=%fromcert and rightid=%fromcert.  I tried changing leftid to explicitly match the CA, but that didn't improve anything.

Any ideas? I can pin to 3.23 for now, but it would be nice to be able to keep up with the OS.

Thanks
Matt
Notice: This e-mail is intended solely for use of the individual or entity to which it is addressed and may contain information that is proprietary, privileged and/or exempt from disclosure under applicable law. If the reader is not the intended recipient or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. This communication may also contain data subject to U.S. export laws. If so, data subject to the International Traffic in Arms Regulation cannot be disseminated, distributed, transferred, or copied, whether incorporated or in its original form, to foreign nationals residing in the U.S. or abroad, absent the express prior approval of the U.S. Department of State. Data subject to the Export Administration Act may not be disseminated, distributed, transferred or copied contrary to U. S. Department of Commerce regulations. If you have received this communication in error, please notify the sender by reply e-mail and destroy the e-mail message and any physical copies made of the communication. Thank you.
JQRSZXPY78973
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libreswan.org/pipermail/swan/attachments/20181218/2c4fa0bf/attachment.html>