[Swan-dev] IPSec offload API

[Steffen Klassert]

On Tue, Nov 29, 2016 at 12:28:13PM -0500, Paul Wouters wrote:
> On Mon, 28 Nov 2016, Ilan Tayari wrote:
> >
> >To sum up, a new XFRM state attribute is introduced: XFRMA_OFFLOAD_DEV
> >Along with the following struct:
> >
> >struct xfrm_user_offload {
> >      int                             ifindex;
> >      __u8                            flags;
> >};
> >#define XFRM_OFFLOAD_IPV6      1
> >#define XFRM_OFFLOAD_INBOUND   2
> >
> >Regarding the swan daemon, we expect the user to configure HW offload explicitly (maybe per-SA, or maybe globally)
> >Then the daemon will apply this attribute to the XFRM states that it wishes to offload.
> 
> Can I ask the use case for this? Why wouldn't hardware offload always be
> used by the kernel when the kernel detects there is such hardware
> support available?

There are several reasons why we want to have it configurable
on a SA basis. First of all, the SADB on the NIC is limited.
So you don't want to offload the first N negotiated SAs, but
those that are expected to have high traffic rates.

Second, the user should be aware what happens. The systems
behaviour changes, a user won't see IPsec packets in the stack
anymore.

Third, you might not trust all NICs that can do ESP offload.


> 
> >Note that the offloaded XFRM state needs the daemon to explicitly specify the network interface ifindex, the SA direction
> >(inbound or outbound), and the used L3 protocol (IPv4 or IPv6).
> >A network interface needs to specify the capability flag NETIF_F_HW_ESP in order to indicate support of ESP crypto offload.
> >All of this may require some extra logic from the daemon.
> 
> So is this a new layer of offloading that works separately from any
> existing hardware offload?

No, it's not really new. It works similar to other network protocol
offloads.

> Does it only apply to nic cards? What if
> someone has some dedicated non-NIC PCI offloading card like the
> Intel QuickAssist?

This is something completely different. Intel QuickAssist does pure
crypto offload. It is (as far as I know) already supported in the
crypto layer.

> 
> >Lastly, the offloaded XFRM state may be rejected by the driver or kernel, by returning an error from the NEWSA message.
> >In such a case, the daemon may wish to fall back to non-offloaded XFRM state, or do something else (configurable?)
> 
> So we install the SA, but it fails and we have to modify and re-install
> the SA?

No, we don't. The current implementation installs the SA to software
if the requested hardware offload fails without any user interaction.

We probably make this behaviour configurable with come flag in a
second step, because some users might not want a software fallback
if the hardware offload fails.

> 
> How well does this offloading deal with out-of-order packets and PFS?

We don't get packet reordering with this, it works like it does
without offloading. Do you refer to Perfect Forward Secrecy with PFS?
If so, the NIC just handles the ESP packet path. It is not involved
in the key negotiation. It takes the key from the key deamon, uses
it and deletes it on request. So I guess there is no impact on this.

> 
> Is there a performance difference/limitation when using one IPsec SA
> versus having many IPsec SA's

Most cpu intensive operations are done by the NIC. So it is not
a big problem anymore that we have to keep a one SA on one cpu.

> (current CPU/AESNI acceleration has
> limits of putting one SA on one CPU only, and if you tweak it to
> use multiple CPU's the out-of-order problem causes so many problems
> the SA ends up getting less throughput)

Btw. if you want to spread the crypto operations for one SA over
multiple cpus, you can use the pcrypt template. It sends the crypto
request to a configurable set of cpus and guarantees to return
the crypto requests in the same order they entered. So you can
have crypto with multiple cpus without packet reordering.

Steffen