[Swan-dev] interop problem between pluto and strongswan with AES128-AES_XCBC-MODP2048

[Andrew Cagney]

Strongswan receives Ni:

Mar 26 23:40:04 14[ENC] parsing NONCE payload, 100 bytes left
...
Mar 26 23:40:04 14[ENC]   parsing rule 10 CHUNK_DATA
Mar 26 23:40:04 14[ENC]    => 32 bytes @ 0x7fc800002330
Mar 26 23:40:04 14[ENC]    0: 69 75 6C C9 B5 F5 68 1C 4D AC 18 68 D7
8D 75 61  iul...h.M..h..ua
Mar 26 23:40:04 14[ENC]   16: 78 3F FB E6 B5 A4 BE F0 8A A3 2A AE 7C
5D 49 5D  x?........*.|]I]
Mar 26 23:40:04 14[ENC] parsing NONCE payload finished

Strongswan emits Nr:

Mar 26 23:40:04 14[ENC] generating payload of type NONCE
///
Mar 26 23:40:04 14[ENC]   generating rule 10 CHUNK_DATA
Mar 26 23:40:04 14[ENC]    => 32 bytes @ 0x7fc800006520
Mar 26 23:40:04 14[ENC]    0: 8D FC D7 85 AF 71 3C 46 1A CB CE D5 1A
A1 2C 0D  .....q<F......,.
Mar 26 23:40:04 14[ENC]   16: E6 AA 2A 14 2E 2E 19 3A 84 02 5C 0C C2
13 21 42  ..*....:..\...!B
Mar 26 23:40:04 14[ENC] generating NONCE payload finished

Pluto forms that into Ni|Nr:

| ike sa SKEYSEED PRF aes_xcbc init Ni|Nr-chunk at 0x7fa9be9defb8 (length 64)
|     ephemeral-key at 0x7fa9c00daf80, size: 16 bytes, type/mechanism: AES_KEY_GEN
...
|     data-bytes at 0x7fa9be9defb8 (64 bytes)
|       69 75 6c c9  b5 f5 68 1c  4d ac 18 68  d7 8d 75 61
|       78 3f fb e6  b5 a4 be f0  8a a3 2a ae  7c 5d 49 5d
|       8d fc d7 85  af 71 3c 46  1a cb ce d5  1a a1 2c 0d
|       e6 aa 2a 14  2e 2e 19 3a  84 02 5c 0c  c2 13 21 42

And feeds that into PRF(Ni|Nr, G^IR), since the key is too big it
rehashes using K=0 before hashing the shared secret:

| XCBC: Key 64<16 too big, rehashing to size
...
| XCBC: data  69 75 6c c9  b5 f5 68 1c  4d ac 18 68  d7 8d 75 61
| XCBC: data  78 3f fb e6  b5 a4 be f0  8a a3 2a ae  7c 5d 49 5d
| XCBC: data  8d fc d7 85  af 71 3c 46  1a cb ce d5  1a a1 2c 0d
| XCBC: data  e6 aa 2a 14  2e 2e 19 3a  84 02 5c 0c  c2 13 21 42
...
| XCBC: K:  00 00 00 00  00 00 00 00  00 00 00 00  00 00 00 00
| XCBC: K1  e1 4d 5d 0e  e2 77 15 df  08 b4 15 2b  a2 3d a8 e0
....

On strongswan (strongswan-5.6.0?) if I'm reading the output from the
attached patch correctly, seems to use the first 8 bytes of NI and Nr:

Mar 26 23:40:04 14[LIB] XCBC: Setting key => 16 bytes @ 0x7fc800003450
Mar 26 23:40:04 14[LIB]    0: 69 75 6C C9 B5 F5 68 1C 8D FC D7 85 AF
71 3C 46  iul...h......q<F

Above is 69 75 6c c9  b5 f5 68 1c from Ni and 8d fc d7 85  af 71 3c 46
from Nr leading to ...

Mar 26 23:40:04 14[LIB] XCBC: Key is just right
Mar 26 23:40:04 14[LIB] XCBC: K => 16 bytes @ 0x7fc800003450
Mar 26 23:40:04 14[LIB]    0: 69 75 6C C9 B5 F5 68 1C 8D FC D7 85 AF
71 3C 46  iul...h......q<F
Mar 26 23:40:04 14[LIB] XCBC: k1 => 16 bytes @ 0x7fc8238a0a90
Mar 26 23:40:04 14[LIB]    0: 97 4E 06 B2 C1 CB 22 82 F6 95 69 EA 98
B3 94 00  .N...."...i.....
Mar 26 23:40:04 14[LIB] XCBC: k2 => 16 bytes @ 0x7fc800002810
Mar 26 23:40:04 14[LIB]    0: 1A 56 1E 60 62 88 79 A3 67 C7 F8 F6 80
F0 E3 04  .V.`b.y.g.......
Mar 26 23:40:04 14[LIB] XCBC: k3 => 16 bytes @ 0x7fc800003270
Mar 26 23:40:04 14[LIB]    0: F3 0E C0 35 BB 01 1D FC 3B A9 64 78 35
71 71 ED  ...5....;.dx5qq.
Mar 26 23:40:04 14[LIB] XCBC: update => 256 bytes @ 0x7fc8000048d0
Mar 26 23:40:04 14[LIB]    0: 6F 77 96 6A F6 A5 4D 06 9F 25 E2 96 D4
C2 D6 E3  ow.j..M..%......
(that's the start of the shared secret)

stumped
-------------- next part --------------
A non-text attachment was scrubbed...
Name: strongswan-xcbc.patch
Type: text/x-patch
Size: 2143 bytes
Desc: not available
URL: <https://lists.libreswan.org/pipermail/swan-dev/attachments/20180327/c39a9edb/attachment.bin>