On 02/02/16 07:52, Jakob Bohm wrote: > I am trying to upgrade an existing 3rd party multithreaded server > from OpenSSL 1.0.2c to 1.0.2f . However when I do so, it starts > mishandling the close_notify "alert". > > 1.0.2f seems to send the close_notify alert unencrypted followed > by an encrypted decrypt_failed alert, where 1.0.2c correctly > sends just an encrypted close_notify alert. > > I am unsure if this exposed a bug in the daemon or in OpenSSL > itself. I have a theory. Previous versions of 1.0.2 handled an SSL_shutdown() call while in the middle of a handshake by ignoring it and returning 1 immediately (meaning everything shutdown successfully). Clearly everything did not shutdown successfully so the return value is not correct. 1.0.2f "fixed" this to attempt to more appropriately shutdown the connection. It now attempts to send a close_notify alert immediately (and will return 0 - meaning more work to do), it will then fail with a -1 return on a second call to SSL_shutdown (meaning we failed to complete a bi-directional shutdown). The reason is that completing a full bi-directional shutdown while mid-handshake is problematic (e.g. if the peer has sent a CCS before it receives our close_notify). My theory as to what is happening in this case is that SSL_shutdown is being called mid-handshake. Could that be the case do you think? It doesn't entirely match with the description you gave (i.e. 1.0.2c would not send an encrypted close_notify...it wouldn't send anything in this scenario)...but its the only change I can think of that might have a bearing on this. If so then I could see a situation where the server has not yet processed a CCS from the client and SSL_shutdown is called. It therefore sends its close_notify in the clear. In the meantime the client has sent its CCS before processing the close_notify. The client then expects the next message from the server to be encrypted...which it obviously isn't. It's not entirely clear to me what the correct behaviour is in that situation, but it is certainly unexpected and a change from what happened before. It looks to me that trying to send a close_notify mid-handshake was probably a bad idea. Perhaps a better fix is just to have SSL_shutdown() return -1 and do nothing if called mid-handshake. I've attached a 1.0.2 patch that does this. Would you be able to try it, and see if that corrects your problem? Thanks Matt -------------- next part -------------- A non-text attachment was scrubbed... Name: shutdown-1.0.2.patch Type: text/x-patch Size: 3888 bytes Desc: not available URL: <http://mta.openssl.org/pipermail/openssl-users/attachments/20160202/89ed97fd/attachment-0001.bin>
