[Swan] RSA key length or ID matching issue in 3.25 FIPS mode

[Paul Wouters]

On Tue, 18 Dec 2018, Matt Hilt wrote:

> I'm running opportunistic encryption between a number of servers all using RHEL 7 with FIPS mode enabled. Everything
> has been working fine using some RSA keys and libreswan 3.23.  RHEL now has 3.25 available, but when upgrading it
> warned that the RSA bit length was required to be > 3072.
> 
> No problem - I switched to some 4096 bit RSA keys and again everything worked fine on 3.23. However, 3.25 is again
> complaining. The errors come in two forms:
> 
> 3.23 <--> 3.25
> 
>   - The 3.25 system still gives the following errors:
> 
>         FIPS: Rejecting cert with key size under 3072

> When running `ipsec whack --listall` I clearly see that the certs and each of its trust chain CAs are all 4096 bit
> RSA. Each server also reports "has private key" for their own cert.

> Any ideas? I can pin to 3.23 for now, but it would be nice to be able to keep up with the OS.

I'll have a look at this and get back to you. We do FIPS testing on RHEL
releases, so I am little confused why you would see an issue that we
did not see. Possibly some X.509 was not properly tested under FIPS?

Paul