[Swan-dev] Marking never pass tests with WIP

Andrew Cagney andrew.cagney at gmail.com
Thu Mar 9 16:23:12 UTC 2017


Looking at http://testing.libreswan.org/results/testing/ and excluding
WIP we seem to consistently have 100 failing tests (you can eyeball
this from the graph) so I did a little analysis using the script
testing/web/never-passed.sh:

- 45 are 'original' and have never ever passed
- 16 are more recent yet also show no sign of passing

I'm going to mark the first 45 as wip.

Andrew

PS: here's the full list with colums <#passes> <#fails>
<#times-tested> sorted by #passes

0 230 230 ikev2-21-mode-mismatch-01
0 230 230 ikev2-45-impair-gx-02
0 230 230 ikev2-allow-narrow-01
0 230 230 ikev2-allow-narrow-02
0 230 230 ikev2-allow-narrow-03
0 230 230 ikev2-ddns-02
0 230 230 interop-ikev2-racoon-02-psk-responder
0 230 230 interop-ikev2-strongswan-19-x509-res-certreq
0 230 230 interop-ikev2-strongswan-21-transport-02
0 230 230 interop-ikev2-strongswan-21-transport-03
0 230 230 interop-ikev2-strongswan-26-ke-mismatch-responder
0 230 230 interop-ikev2-strongswan-27-fragmentation
0 230 230 newoe-07-ike-replace-initiator-esp
0 230 230 newoe-15-portpass
0 230 230 newoe-18-poc-block
0 230 230 newoe-18-poc-blockall
0 230 230 newoe-18-private-clear
0 230 230 newoe-18-private-clearall
0 230 230 newoe-19-poc-poc-clear
0 230 230 newoe-20-ipv6
0 230 230 newoe-21-liveness-clear
0 230 230 newoe-25-cat-2
0 230 230 nss-cert-08-mismatch
0 230 230 nss-cert-nosecret
0 230 230 nss-cert-ocsp-01-strict
0 230 230 nss-cert-ocsp-02
0 230 230 nss-cert-ocsp-02-ikev2
0 230 230 nss-cert-ocsp-03-strict
0 230 230 nss-cert-ocsp-04
0 230 230 nss-cert-ocsp-05-strict
0 230 230 nss-cert-ocsp-06
0 230 230 nss-cert-ocsp-07-nourl
0 230 230 x509-pluto-frag-01
0 230 230 xauth-pluto-17
0 230 230 xauth-pluto-18
0 218 218 dnssec-pluto-01
0 218 218 dpd-01
0 218 218 dpd-04
0 218 218 dpd-06
0 218 218 l2tp-01
0 218 218 l2tp-02
0 218 218 nat-pluto-04
0 218 218 nflog-03-conns
0 177 177 seccomp-01-enabled
0 177 177 seccomp-02-tolerant
0 153 153 interop-ikev2-strongswan-21-transport-04
0 152 152 nss-cert-09-notyetvalid-initiator
0 152 152 nss-cert-09-notyetvalid-initiator-ikev2
0 152 152 nss-cert-10-notyetvalid-responder
0 152 152 nss-cert-10-notyetvalid-responder-ikev2
0 152 152 nss-cert-ocsp-08-post
0 75 75 addconn-01
0 53 53 certoe-01-whack
0 53 53 certoe-07-nat-2-clients
0 53 53 certoe-08-nat-packet-cop-restart
0 49 49 nss-cert-chain-01-ikev2
0 49 49 nss-cert-chain-03-ikev2
0 49 49 nss-cert-chain-04-ikev2
0 37 37 ikev1-aggr-sendcert-01
0 26 26 ikev2-child-ipseac-replace-ike
0 18 18 netkey-vti-06c
1 229 230 newoe-25-cat-3-4-way
2 228 230 ikev2-27-uniqueid
2 3 5 ikev2-child-ipsec-responder
4 11 15 ikev2-54-replace-leave-old
5 225 230 nflog-01-global
6 224 230 ipsec-hostkey-ckaid-02-fips
6 224 230 nss-cert-05
8 0 8 ikev2-algo-ike-dh-ecp-01
8 0 8 interop-ikev2-strongswan-33-ike-dh-ecp-responder
10 8 18 netkey-vti-05
11 28 39 interop-ikev2-strongswan-31-asymmetric-responder
12 139 151 nat-pluto-02-klips-klips
13 1 14 ikev2-initiate-template-01
17 1 18 netkey-vti-06
18 200 218 netkey-vti-01
19 132 151 nat-pluto-02-netkey-klips
24 194 218 ikev2-liveness-05
25 205 230 ikev2-frag-02-ipv6
25 70 95 nss-cert-badca-01
25 1 26 interop-ikev2-strongswan-15-child-sa
27 203 230 ikev2-31-nat-rw-no-rekey
31 199 230 newoe-25-cat-4
33 197 230 interop-ikev2-strongswan-23-initiator-cp
33 197 230 nss-cert-ocsp-01
33 197 230 nss-cert-ocsp-01-chain
33 197 230 nss-cert-ocsp-01-ikev2
33 197 230 nss-cert-ocsp-03
33 197 230 nss-cert-ocsp-03-ikev2
33 197 230 nss-cert-ocsp-05
33 197 230 nss-cert-ocsp-05-ikev2
33 185 218 compress-pluto-01
33 185 218 compress-pluto-02
34 2 36 ikev1-27-uniqueid
35 70 105 klips-passthrough-00
35 1 36 ikev1-aggr-replace-01
39 191 230 newoe-25-cat-1
39 14 53 certoe-06-nat-packet-cop
41 189 230 ikev2-12-x509-ikev1-rw
41 12 53 certoe-03-cop-whack
41 12 53 certoe-03-poc-whack
41 6 47 delete-sa-09-rhbz1311360-c32
42 11 53 certoe-04-poc-packet
45 8 53 ikev2-asymmetric-01-parsing
47 1 48 delete-sa-08-rhbz1311360
48 5 53 ikev2-asymmetric-03-null-rsacert
48 1 49 nss-cert-chain-02-ikev2
48 1 49 nss-cert-chain-04
49 4 53 ikev2-asymmetric-02-orienting
49 4 53 ikev2-asymmetric-10-rsaraw-psk
50 180 230 ikev2-09-rw-rsa
50 83 133 ikev2-16-alias-whack-up
50 3 53 ikev2-asymmetric-04-null-rsaraw
50 3 53 ikev2-asymmetric-07-psk-rsaraw
50 3 53 ikev2-asymmetric-08-psk-rsacert
50 3 53 ikev2-asymmetric-09-rsaraw-null
50 3 53 ikev2-asymmetric-12-rsacert-null
50 3 53 ikev2-asymmetric-13-rsacert-psk
50 3 53 ikev2-asymmetric-15-rsaraw-rsaraw
52 116 168 interop-ikev2-strongswan-09-dh22-initiator
52 1 53 certoe-02-whack-badca
52 1 53 certoe-10-symetric-cert-whack
52 1 53 certoe-11-symetric-cert-nat
52 1 53 ikev2-asymmetric-16-auth-mismatch
52 1 53 multinet-04
53 0 53 certoe-05-poc-reverse
57 152 209 delete-sa-04
63 167 230 ikev2-01-fallback-ikev1
66 39 105 klips-algo-twofish-01
67 163 230 nss-cert-crl-02
71 34 105 klips-algo-serpent-01
72 158 230 interop-ikev2-strongswan-03-psk-initiator
72 158 230 interop-ikev2-strongswan-28-reauth
72 158 230 nss-cert-chain-02
72 100 172 ikev1-impair-gx-02
72 100 172 psk-pluto-02
73 157 230 nss-cert-04
74 156 230 fips-08-ikev2-x509
81 149 230 interop-ikev2-strongswan-07-strongswan
83 147 230 interop-ikev1-strongswan-08-strongswan-cast
87 143 230 nss-cert-crl-03-strict
89 16 105 klips-algo-cast-01
90 140 230 nss-cert-crl-03
90 15 105 klips-basic-pluto-01
94 124 218 klips-netkey-pluto-06
94 124 218 netkey-klips-pluto-03
105 125 230 fips-03-ikev1-md5
105 26 131 ikev2-53-clean-pending
109 121 230 interop-ikev2-strongswan-04-x509-responder
110 41 151 nat-pluto-02-klips-netkey
113 117 230 interop-ikev2-strongswan-21-transport-01
113 105 218 netkey-audit-01
117 113 230 ikev2-liveness-03
117 101 218 ikev1-algo-ike-bogus-01
118 112 230 interop-ikev2-strongswan-22-cp-responder-psk
120 3 123 ikev2-48-nat-cp-start
121 47 168 interop-ikev1-strongswan-09-dh22-initiator
126 104 230 ikev2-algo-sha2-06
128 102 230 newoe-08-ike-replace-responder
132 98 230 nss-cert-crl-01-strict
140 78 218 nat-pluto-03
140 69 209 ikev2-10-2behind-nat
145 85 230 aggr-pluto-01
146 72 218 algo-pluto-08
149 81 230 newoe-07-ike-replace-initiator
150 80 230 ikev2-48-nat-cp
150 80 230 newoe-25-cat-5
151 79 230 nflog-02-conn
151 79 230 nss-cert-chain-01
153 77 230 nss-cert-02-ikev2
154 76 230 nss-cert-chain-03
155 75 230 nss-cert-07
159 59 218 algo-pluto-11
159 59 218 ikev1-algo-ike-aes-01
159 59 218 netkey-algo-twofish-01
160 58 218 netkey-algo-aes_ccm-01
160 58 218 netkey-algo-aes_gcm-01
160 58 218 netkey-algo-aes_gcm-02
160 58 218 netkey-algo-aes_gcm-03
161 57 218 ikev1-algo-ike-aes-02
167 63 230 dpd-05
167 63 230 ikev2-delete-02
167 63 230 netkey-tfc-01
167 63 230 netkey-tfc-02
168 62 230 ikev2-algo-ike-sha2-02
168 62 230 nss-cert-crl-01
169 61 230 aggr-pluto-02
170 48 218 nat-pluto-02
171 59 230 netkey-tfc-03
171 47 218 ah-pluto-07-klips-netkey
172 58 230 ikev2-invalid-ke-02-wrong-modp
174 56 230 ikev2-11-simple-psk
174 56 230 ikev2-algo-ike-sha2-03
174 56 230 interop-ikev2-strongswan-02-psk-responder
174 56 230 xauth-pluto-05
174 56 230 xauth-pluto-06
174 56 230 xauth-pluto-09
174 56 230 xauth-pluto-19
175 55 230 basic-pluto-06
175 55 230 ikev2-algo-07-aes_ctr
175 55 230 nss-cert-01-ikev2
175 55 230 xauth-pluto-05-netkey
177 53 230 fips-default-ikev2-01-nofips-east
177 53 230 ikev2-03-basic-rawrsa
177 53 230 ikev2-29-no-rekey
177 53 230 ikev2-47-priority
177 53 230 ikev2-algo-04-aes-gcm256
177 53 230 ikev2-algo-06-aes-aes_xcbc
177 53 230 ikev2-algo-sha2-08
177 53 230 ikev2-delete-01
177 53 230 interop-ikev2-strongswan-25-ke-mismatch-initiator
178 52 230 ikev2-41-rw-replace
178 52 230 ikev2-42-rw-replace-responder
178 52 230 ikev2-algo-03-aes-ccm
178 52 230 ikev2-algo-10-aes-gcm128
178 52 230 ikev2-algo-sha2-01
178 52 230 ikev2-algo-sha2-02
178 52 230 ikev2-algo-sha2-03
178 52 230 ikev2-algo-sha2-04
178 52 230 ikev2-allow-narrow-04
178 40 218 basic-pluto-03
179 51 230 basic-pluto-13-netkey-route
179 51 230 ikev2-04-basic-x509
179 51 230 ikev2-04-basic-x509-ckaid
179 51 230 ikev2-04-basic-x509-nhelpers0
179 51 230 ikev2-04-basic-x509-nsspw
179 51 230 ikev2-algo-11-gcm-prop2
179 51 230 ikev2-algo-ike-sha2-01
179 51 230 ikev2-allow-narrow-05
179 51 230 ikev2-allow-narrow-06
179 51 230 ikev2-allow-narrow-07
179 39 218 algo-pluto-10
179 39 218 basic-pluto-01
179 39 218 ikev1-algo-ike-sha2-01
179 39 218 ikev1-algo-ike-sha2-02
179 39 218 ikev1-algo-ike-sha2-03
179 39 218 ikev2-17-rekey-ipsec
179 39 218 netkey-algo-null-01
179 39 218 netkey-algo-serpent-01
180 50 230 ikev2-12-transport-psk
180 50 230 newoe-22-nat-poc-cop
181 49 230 x509-pluto-02
181 49 230 x509-pluto-03
181 37 218 algo-pluto-01
181 37 218 basic-pluto-02
181 37 218 ikev1-algo-esp-null-01
181 37 218 ikev1-cryptoload-00
181 37 218 ikev1-isakmp-reserved-flags-01
181 37 218 ikev1-isakmp-reserved-flags-02
182 48 230 nss-cert-03-ikev2
182 36 218 compress-pluto-netkey-03
182 36 218 delete-sa-03
183 47 230 fips-04-ikev2-md5
183 47 230 x509-pluto-01
183 35 218 compress-pluto-netkey-klips-04
183 35 218 ipv6-tunnel-mode-04-rw
184 46 230 algo-pluto-13-invalid-3des
184 46 230 ikev1-algo-esp-sha2-05
184 46 230 ikev2-12-x509-ikev1
184 46 230 ikev2-21-mode-mismatch-02
184 46 230 ikev2-algo-sha2-05
184 46 230 interop-ikev2-strongswan-05-psk-md5
184 46 230 newoe-10-expire-inactive-ike
184 46 230 nss-cert-01
184 46 230 nss-cert-02
184 46 230 nss-cert-02-eku
184 46 230 nss-cert-02-eku-combined
184 46 230 nss-cert-03
184 46 230 nss-cert-04-ikev2
184 46 230 nss-cert-05-ikev2
184 46 230 nss-cert-06
185 45 230 ikev2-liveness-01
186 44 230 ikev1-algo-esp-sha2-01-netkey-klips
187 43 230 fips-07-ikev2-3des-sha256
187 43 230 ikev2-50-propnum
188 42 230 interop-ikev1-strongswan-02-psk-responder
188 42 230 interop-ikev2-strongswan-06-aes192
188 42 230 newoe-10-expire-inactive
188 30 218 netkey-algo-null-02
188 28 216 ipv6-tunnel-mode-02-netkey-netkey
193 37 230 newoe-09-mutual
194 36 230 newoe-06-prio
195 35 230 ikev2-dcookie-01
197 33 230 ikev2-30-rw-no-rekey
197 33 230 interop-ikev2-strongswan-12-nat-responder
199 31 230 ikev1-algo-esp-sha2-02-netkey-klips
199 31 230 newoe-04-pass-pass
201 29 230 fips-09-ikev2-gcm
201 29 230 ikev2-invalid-ke-01-invalid-modp
201 29 230 ikev2-invalid-ke-03-default-initiator
201 29 230 ikev2-invalid-ke-04-default-responder
202 28 230 interop-ikev2-strongswan-20-psk-aes_xcbc
202 28 230 interop-ikev2-strongswan-24-aes_ctr
204 26 230 interop-ikev1-strongswan-15-twofish
204 14 218 xauth-pluto-20-pam
204 5 209 ikev2-dcookie-02
204 5 209 ikev2-dcookie-03
205 13 218 xauth-pluto-14
206 24 230 psk-pluto-06
206 12 218 dynamic-iface-01
207 11 218 fips-06-ikev1-3des-sha1
208 10 218 algo-pluto-12-aes-default
209 21 230 interop-ikev1-strongswan-16-serpent
209 9 218 ah-pluto-01
210 8 218 ah-pluto-02
210 8 218 ah-pluto-03
210 8 218 ah-pluto-04
210 8 218 ah-pluto-05
210 8 218 ah-pluto-11
210 8 218 nat-pluto-07
210 8 218 nat-transport-02
211 7 218 pluto-ipcmp-01
211 7 218 pluto-rekey-02
211 7 218 xauth-pluto-03
212 18 230 ikev2-03-basic-rawrsa-nhelpers0
212 18 230 ikev2-algo-05-aes-default
212 18 230 ikev2-algo-esn-04
212 18 230 ikev2-isakmp-reserved-flags-01
212 18 230 ikev2-isakmp-reserved-flags-02
212 18 230 ikev2-minor-version-initiator
212 18 230 ikev2-minor-version-responder
212 18 230 ikev2-no-nhelpers-01
212 18 230 ikev2-payload-reserved-flags-01
212 18 230 newoe-05-hold-pass
213 17 230 algo-pluto-07
213 17 230 fips-default-ikev2-02-nofips-west
213 17 230 ikev2-algo-esn-01
213 17 230 ikev2-algo-esn-03
213 17 230 ikev2-algo-esn-05
213 17 230 ikev2-algo-esn-06
213 5 218 dpd-03
213 5 218 ikev1-algo-ike-pfs-01
213 5 218 l2tp-02-netkey
214 16 230 ikev2-32-nat-rw-rekey
214 4 218 ikev2-02-responder-send-notify
214 4 218 netkey-algo-cast-01
214 4 218 netkey-vti-02
215 15 230 xauth-pluto-07
215 3 218 algo-pluto-14-aes_ctr
215 3 218 dpd-02
215 3 218 fips-default-ikev1-01-nofips-east
215 3 218 fips-default-ikev1-02-nofips-west
215 3 218 netkey-algo-camellia-01
215 3 218 netkey-passthrough-02
216 14 230 dpd-07
216 14 230 xauth-pluto-08
216 14 230 xauth-pluto-13
216 14 230 xauth-pluto-15
216 14 230 xauth-pluto-16
216 2 218 delete-sa-01
217 13 230 ikev2-algo-esn-02
217 1 218 ah-pluto-10
217 1 218 listen-change-01
217 1 218 netkey-passthrough-00
217 1 218 netkey-passthrough-01
217 1 218 netkey-passthrough-03
217 1 218 netkey-passthrough-ipxfrm
218 12 230 ikev2-25-rw-nat
218 12 230 ikev2-algo-09-camellia
218 12 230 ikev2-liveness-02
218 12 230 interop-ikev1-strongswan-14-camellia
219 11 230 ikev2-13-ah
219 11 230 nat-pluto-01
219 11 230 xauth-pluto-12
220 10 230 ikev2-46-basic-psk-netkey
221 9 230 ikev2-liveness-06
222 8 230 ikev2-liveness-07
222 8 230 ikev2-x509-01
223 7 230 nat-pluto-08
223 7 230 nat-pluto-09
223 7 230 xauth-pluto-04
224 6 230 ikev2-05-basic-psk
224 6 230 ikev2-05-basic-psk-nhelpers0
224 6 230 ikev2-08-delete-notify
224 6 230 ikev2-38-psk-65bytes
224 6 230 ikev2-39-psk-64bytes
224 6 230 ikev2-43-init-retransmit
224 6 230 ikev2-algo-01-modp2048-initiator
224 6 230 ikev2-algo-02-modp2048-responder
224 6 230 ikev2-frag-01
225 5 230 ikev2-algo-08-cast
225 5 230 ikev2-auth-null-01
225 5 230 interop-ikev2-strongswan-05-psk-aes
225 5 230 interop-ikev2-strongswan-09-psk-aes-ccm
225 5 230 interop-ikev2-strongswan-09-psk-aes-gcm
225 5 230 interop-ikev2-strongswan-17-delete-sa-responder
225 5 230 interop-ikev2-strongswan-18-psk-cast
225 5 230 interop-ikev2-strongswan-30-esn
225 5 230 psk-pluto-05
226 4 230 ikev1-algo-05-3des-sha2
226 4 230 interop-ikev1-strongswan-04-psk-aes-ccm
226 4 230 interop-ikev1-strongswan-04-psk-aes-gcm
226 4 230 interop-ikev2-strongswan-10-nat-initiator
226 4 230 interop-ikev2-strongswan-11-nat-initiator
226 4 230 psk-pluto-04
227 3 230 ikev2-14-missing-ke
227 3 230 ikev2-15-fuzzer
227 3 230 x509-pluto-frag-04
228 2 230 ikev1-01-fuzzer
228 2 230 ikev1-impair-gx-01
228 2 230 ikev2-45-impair-gx-01
228 2 230 ikev2-ddns-01
228 2 230 interop-ikev1-strongswan-12-esp-sha2_256
228 2 230 interop-ikev2-strongswan-13-ah-initiator
228 2 230 interop-ikev2-strongswan-16-ah-initiator-sha512
228 2 230 newoe-01-whack
228 2 230 newoe-18-poc-cop
228 2 230 psk-pluto-01
228 2 230 x509-pluto-frag-02
229 1 230 ah-pluto-09
229 1 230 ikev1-algo-04-mismatch
229 1 230 ikev1-cryptoload-01
229 1 230 ikev2-major-version-initiator
229 1 230 ikev2-major-version-responder
229 1 230 interop-ikev1-strongswan-03-psk-initiator
229 1 230 interop-ikev1-strongswan-07-cast128
229 1 230 interop-ikev1-strongswan-10-ah-initiator-sha256
229 1 230 interop-ikev1-strongswan-11-ah-initiator-sha512
229 1 230 interop-ikev1-strongswan-13-esp-sha2_512
229 1 230 interop-ikev2-strongswan-14-delete-sa
229 1 230 interop-ikev2-strongswan-17-ah-initiator-sha256
229 1 230 ipsec-hostkey-ckaid-03-nsspw
229 1 230 newoe-08-restart
229 1 230 newoe-12-clear
229 1 230 newoe-13-block
229 1 230 newoe-14-clear-in-block-clear
229 1 230 newoe-16-pass-hold
229 1 230 newoe-18-block-block
229 1 230 newoe-18-block-clear
229 1 230 newoe-18-block-cop
229 1 230 newoe-18-block-poc
229 1 230 newoe-18-block-private
229 1 230 newoe-18-clear-block
229 1 230 newoe-18-clear-clear
229 1 230 newoe-18-clear-cop
229 1 230 newoe-18-clear-poc
229 1 230 newoe-18-clear-private
229 1 230 newoe-18-cop-block
229 1 230 newoe-18-cop-clear
229 1 230 newoe-18-cop-cop
229 1 230 newoe-18-cop-poc
229 1 230 newoe-18-cop-private
229 1 230 newoe-18-poc-clear
229 1 230 newoe-18-poc-poc
229 1 230 newoe-18-poc-private
229 1 230 newoe-18-private-block
229 1 230 newoe-18-private-cop
229 1 230 newoe-18-private-copall
229 1 230 newoe-18-private-poc
229 1 230 newoe-18-private-pocall
229 1 230 newoe-18-private-private
229 1 230 newoe-23-reverse-nat
229 1 230 psk-pluto-03
229 1 230 replay-authip-01
229 1 230 x509-pluto-frag-00
229 1 230 x509-pluto-frag-03
230 0 230 ipsec-hostkey-ckaid-01
230 0 230 newoe-03-oeself
230 0 230 newoe-11-failike


More information about the Swan-dev mailing list