[Swan-dev] IPSec offload API

[Steffen Klassert]

On Wed, Nov 30, 2016 at 08:41:41AM -0500, Paul Wouters wrote:
> On Wed, 30 Nov 2016, Steffen Klassert wrote:
> 
> >>Can I ask the use case for this? Why wouldn't hardware offload always be
> >>used by the kernel when the kernel detects there is such hardware
> >>support available?
> >
> >There are several reasons why we want to have it configurable
> >on a SA basis. First of all, the SADB on the NIC is limited.
> >So you don't want to offload the first N negotiated SAs, but
> >those that are expected to have high traffic rates.
> 
> You say these drivers have a limit to the number of concurrent SA's
> the can support - as opposed to a limit to the number of packets
> they can process? Where does that limit come from? Does the private
> IPsec SA key get pushed into the card? 

Yes.

> If so, then we also have FIPS
> issues to deal with - that is either those drivers need to come
> inside the crypto boundary or we have to disable offload in FIPS mode.

I'm not so familiar when FIPS mode allows, but
be probably have to disable offloading in FIPS mode.

> 
> >Second, the user should be aware what happens. The systems
> >behaviour changes, a user won't see IPsec packets in the stack
> >anymore.
> 
> The behaviour I was expecting was IPsec processing always works. But
> it either gets offloaded or not.

IPsec processing works always, but you don't have encrypted IPsec
packets in the stack anymore because the NIC does this, before
the packet enters the stack on receive and after the packets leaves
the stack on transmit.

> 
> >Third, you might not trust all NICs that can do ESP offload.
> 
> It seems like an unlikely corner case that you would trust some
> NICs in your machine but not some others.

This is a valid case in high security environments. The NIC
has the key and can do whatever it want with it. So somebody
might just trust the NIC if he knows the crypto code that is
running on the NIC.

> 
> >>Does it only apply to nic cards? What if
> >>someone has some dedicated non-NIC PCI offloading card like the
> >>Intel QuickAssist?
> >
> >This is something completely different. Intel QuickAssist does pure
> >crypto offload. It is (as far as I know) already supported in the
> >crypto layer.
> 
> It is. I am just wondering how these different methods will interact,
> or at least not interfere with each other.

If we do crypto offload to the NIC, we don't use the crypto layer.
We do the ESP encapsulation in the stack and defer the crypto
operations to the NIC by just not calling the crypto layer.

> 
> >We probably make this behaviour configurable with come flag in a
> >second step, because some users might not want a software fallback
> >if the hardware offload fails.
> 
> I think that is highly unlikely.

This is also a valid case in high security environments. In
some cases the crypto operation _must_ be processed by a
special NIC, software crypto or in particular AESNI is not
allowed.

> >
> >Most cpu intensive operations are done by the NIC. So it is not
> >a big problem anymore that we have to keep a one SA on one cpu.
> 
> On a multi core machine, using multiple CPU's could yield a better
> performance than offloading it to a single nic using a single CPU.

As Ilan mentioned, this NIC offloading can be magnitudes faster
than everything you can get with spreading traffic of one SA over
multiple cores.

> 
> >Btw. if you want to spread the crypto operations for one SA over
> >multiple cpus, you can use the pcrypt template. It sends the crypto
> >request to a configurable set of cpus and guarantees to return
> >the crypto requests in the same order they entered. So you can
> >have crypto with multiple cpus without packet reordering.
> 
> In almost all cases I've seen, pcrypt makes things worse unless you
> completely disable replay protection (using replay-window=0)

I have the opposite experience here. We Don't get any packet reorder
with that.

Steffen