On Tue, 2017-08-15 at 11:42 -0700, Daniel Lenski wrote:
>
> ?#define ESP_OVERHEAD (4 /* SPI */ + 4 /* sequence number */ + \
> -???????? 20 /* biggest supported MAC (SHA1) */ + 16 /* biggest supported IV (AES-128) */ + \
> -??????? 1 /* pad length */ + 1 /* next header */ + \
> -???????? 16 /* max padding */ )
> +????????????????????? 1 /* pad length */ + 1 /* next header */ + \
> +????????????????????? 16 /* max padding */ )
> ?#define UDP_HEADER_SIZE 8
> ?#define IPV4_HEADER_SIZE 20
> ?#define IPV6_HEADER_SIZE 40
> @@ -323,7 +322,9 @@ static int calculate_mtu(struct openconnect_info *vpninfo)
> ?
> ????????if (!mtu) {
> ????????????????/* remove IP/UDP and ESP overhead from base MTU to calculate tunnel MTU */
> -???????????????mtu = base_mtu - ESP_OVERHEAD - UDP_HEADER_SIZE;
> +???????????????mtu = ( base_mtu - UDP_HEADER_SIZE - ESP_OVERHEAD
> +?????????????????????? - (vpninfo->hmac_key_len ? : 20) /* biggest supported MAC (SHA1) */
> +?????????????????????? - (vpninfo->enc_key_len ? : 32) /* biggest supported IV (AES-256) */ );
> ????????????????if (vpninfo->peer_addr->sa_family == AF_INET6)
> ????????????????????????mtu -= IPV6_HEADER_SIZE;
Better... except that you left the padding hard-coded to 16. That would
be the same as the cipher blocksize (== IV size).... except you don't
actually *need* to be pessimistic. It can be calculated exactly, just
as dtls_get_data_mtu() does in openssl-dtls.c
