Hi > -----Original Message----- > From: Gaurav Jain > Sent: Monday, January 10, 2022 1:12 PM > To: John Baldwin <jhb@xxxxxxxxxxx>; borisp@xxxxxxxxxxxx; openssl- > users@xxxxxxxxxxx; matt@xxxxxxxxxxx > Cc: Varun Sethi <V.Sethi@xxxxxxx>; Pankaj Gupta <pankaj.gupta@xxxxxxx> > Subject: RE: [EXT] Re: KTLS with openssl 3.0 fail with error > ENOTCONN(Transport endpoint is not connected) > > Hi John/Matt > > > -----Original Message----- > > From: John Baldwin <jhb@xxxxxxxxxxx> > > Sent: Thursday, January 6, 2022 11:21 PM > > To: Gaurav Jain <gaurav.jain@xxxxxxx>; borisp@xxxxxxxxxxxx; openssl- > > users@xxxxxxxxxxx > > Cc: Varun Sethi <V.Sethi@xxxxxxx>; Pankaj Gupta <pankaj.gupta@xxxxxxx> > > Subject: Re: [EXT] Re: KTLS with openssl 3.0 fail with error > > ENOTCONN(Transport endpoint is not connected) > > > > Caution: EXT Email > > > > On 1/6/22 5:58 AM, Gaurav Jain wrote: > > > Hi > > > > > >> -----Original Message----- > > >> From: John Baldwin <jhb@xxxxxxxxxxx> > > >> Sent: Thursday, January 6, 2022 12:26 AM > > >> To: Gaurav Jain <gaurav.jain@xxxxxxx>; borisp@xxxxxxxxxxxx; > > >> openssl- users@xxxxxxxxxxx > > >> Cc: Varun Sethi <V.Sethi@xxxxxxx>; Pankaj Gupta > > >> <pankaj.gupta@xxxxxxx> > > >> Subject: [EXT] Re: KTLS with openssl 3.0 fail with error > > >> ENOTCONN(Transport endpoint is not connected) > > >> > > >> Caution: EXT Email > > >> > > >> On 1/4/22 11:49 PM, Gaurav Jain wrote: > > >>> Hello Boris/John > > >>> > > >>> I am from NXP and currently working on enabling KTLS on NXP > > >>> platforms via > > >> openssl. > > >>> I see that you enabled KTLS support in openssl > > >> 3.0(https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F > > >> %2 > > >> Fwww > > >> .openssl.org%2Fnews%2Fchangelog.html%23openssl- > > >> > > > 30&data=04%7C01%7Cgaurav.jain%40nxp.com%7Ce87da43a5488475b2aa > > >> > > > d08d9d07d05b0%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C1%7C63777 > > >> > > > 0057654781203%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ > > >> > > > IjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=vZa0aCu > > >> D%2FzrXB0vv23DZiOWSVichep42YLqA4a1JeXY%3D&reserved=0). > > >>> > > >>> when I configure openssl 3.0 or 3.1.0 with enable-ktls and and try > > >>> to run the > > >> s_server, s_client application. > > >>> I observe that connection is successfully established - but it didn't use KTLS. > > >>> > > >>> Then I added additional log in kernel(file net/tls/tls_main.c) and > > >>> see that kernel is returning error -ENOTCONN when (sk->sk_state != > > >>> TCP_ESTABLISHED) in function static int tls_init(struct sock *sk) > > >> > > >> To be clear, I have worked on KTLS support for FreeBSD, not for Linux. > > >> > > >> However, I think the error you are seeing is a red herring. I > > >> think you are seeing the setsockopt() call from ktls_enable() fail > > >> because it is invoked on the listen socket since ktls_enable() is > > >> called when sockets are > > created by libssl. > > >> > > >> For KTLS to work on the server side on Linux what you need to find > > >> out is when > > >> ktls_enable() is invoked on the socket returned by accept() and why > > >> that is > > failing. > > >> > > > > > > Thanks John for your input. > > > Ktls_enable() after accept() is successful on server side. > > > I added debug logs, ktls_start() is failing with error Invalid argument. > > > > > > Logs: > > > openssl s_server -ktls -key rsa.key -cert server.pem -accept 443 > > > > > > Using default temp DH parameters > > > > > > ACCEPT > > > > > > ktls_enable setsockopt success, ret = 0 > > > > > > ktls_enable() = 1 > > > > > > > > > fd = 4, is_tx = 0, tls_crypto_info_len = 1872610871009456445 > > > > > > ktls_start setsockopt failed, 22, Invalid argument > > > > > > fd = 4, is_tx = 2, tls_crypto_info_len = 8329596950154514032 > > > > > > ktls_start setsockopt failed, 22, Invalid argument > > > > You'd have to add traces to see why this is failing I think. I know > > on FreeBSD we will fail the equivalent setsockopt if the ciphersuite isn't > supported (e.g. > > some cipher suites we only support if the socket is routed over a NIC > > with offload support, and there can also be system tunables to disable > > certain cipher suites). > > I don't quite know what that is like on the Linux side though. > > > > -- > > John Baldwin > > I debugged further the invalid argument error reported from setsockopt called in > ktls_start(). > Problem is with crypto_info->tls_crypto_info_len. > when I applied the below change I could run s_server, s_client on linux with KTLS > support. > below patch enables the aes_gcm_128 ciphers 1. AES128-GCM-SHA256 2. > ECDHE-RSA-AES128-GCM-SHA256 > > diff --git a/include/internal/ktls.h b/include/internal/ktls.h index > 95492fd065..30ed5072de 100644 - a/include/internal/ktls.h > +++ b/include/internal/ktls.h > @@ -285,7 +285,7 @@ static ossl_inline int ktls_start(int fd, ktls_crypto_info_t > *crypto_info, int is_tx) { > return setsockopt(fd, SOL_TLS, is_tx ? TLS_TX : TLS_RX, > - crypto_info, crypto_info->tls_crypto_info_len) ? 0 : 1; > + crypto_info, sizeof(crypto_info->gcm128)) ? 0 : > + 1; > } > > When trying to enable other 2 ciphers, I am getting build errors in openssl. > 3. AES256-GCM-SHA384 > 4. ECDHE-RSA-AES256-GCM-SHA384 > when trying to build openssl for gcm256, getting compilation errors: > > In file included from crypto/bio/bio_sock2.c:15: > include/internal/ktls.h:251:46: error: field 'gcm256' has incomplete type > struct tls12_crypto_info_aes_gcm_256 gcm256; > ^~~~~~ > > By default Openssl has support for aes_gcm_128 ciphers. > When include kernel tls.h file in openssl, aes_gcm_256 ciphers can be built and > run without errors. > > I feel that Openssl KTLS support for Linux is broken. > Opened a bug https://github.com/openssl/openssl/issues/17451 On further debugging with Openssl 3.0, observed that BIO_CTRL_SET_KTLS command is not received in sock_ctrl() function. so ktls_start() is never called. > Regards > Gaurav Jain
