On Thu, Jul 09, 2020 at 06:07:41PM +0000, Andreas Müller wrote: > Hi, > > I "inherited" our project to support/use TLSv1.3 from a late colleague. We > have a server written in C++ (Windows, Linux) > and clients (Windows, Linux, also written in C++ and also a Java client). > With Java, we use the native SSLSocket implementation, in Windows we use > Schannel and in Linux we use OpenSSL 1.1.1g. It seems to work and even > interoperability > did not show issues on some initial testing. > > I was curious about SSL_key_update. I read that other implementations use it > automatically, but with OpenSSL I have to > do it manually. So I added a > > int rc = SSL_key_update(ssl, SSL_KEY_UPDATE_REQUESTED); > > after 1000 I/O operations and looked what happened. > > I started with the Java client and it gets wrong application data in such a > situation. > > I tested with our Windows implementation (I know it may have > interoperability issues) and here I can see the > data I get from the server side and it is the same that I see on server side > in the callback, but the Windows > DecryptMessage function fails with SEC_E_INVALID_TOKEN. (I had expected > something like SEC_I_RENEGOTIATE to > get the information to put this record into InitializeSecurityContext.) > > The Linux client also aborts the connection, but here I have not yet more > details, but either the decryption fails > or the decrypted application data is wrong. Hopefully I can dig in there > next week. > > Here is what happens on server side: > > 1. I call SSL_key_update (see above) > 2. I call SSL_write with application data > 3. The write callback is invoked with this data: > DATA: 17 03 03 00 16 b6 02 b1 06 87 f2 30 0d 77 35 31 7a 5b 6d 3c c2 aa > 11 2c 32 95 a9 > 4. The write callback is invoked again with application data provided to > SSL_write: > DATA: 17 03 03 00 45 12 24 e5 66 36 2f 28 ea 62 2e 17 4c 62 f0 38 07 7f > 72 70 87 25 ba 45 ff cf f7 9f 0d 7d 48 ... > > I see these data arrive at the client side (Windows): > DATA: 17 03 03 00 16 b6 02 b1 06 87 f2 30 0d 77 35 31 7a 5b 6d 3c c2 aa > 11 2c 32 95 a9 > but get the error described above. In Java I get wrong application data, so > it seems to decrypt this as application > data?! > > I saw an additional call to SSL_do_handshake in the apps/s_server.c and > tried this, but it did not change anything. > > Is there anything else (except calling SSL_key_update) I have to take care of? Per the documentation (https://www.openssl.org/docs/man1.1.1/man3/SSL_key_update.html) it sounds like you're doing all you need to be. (In particular, you don't need to call SSL_do_handshake().) Probably the fastest way to track down what's happening is to get a full packet capture and keylog (e.g., with s_client -keylogfile path/to/log, then point wireshark at the trace+keys. -Ben
