On Fri, Oct 07, 2016 at 10:30:06PM +0530, Ajay Garg wrote: > Ok, so for sending app-payload-bytes, we do a bio_write() to "bio1", > and if "bio1" requires reading from bio2/peer, bio_write() will return > SSL_ERROR_WANT_READ (even for blocking sockets). We then read-in some > app-payload-encrypted-bytes from device-socket, No, it will *usually* return SSL_ERROR_WANT_WRITE, that's when you write "some" (based on pending data amount from bio2) to the network that you read from bio2. Those writes might be written via a buffer you interpose, however it is important to flush that buffer as much as possible before attempting any network reads (flush fully if the network reads are potentially blocking). However, it might sometimes return SSL_ERROR_WANT_READ, in which case, the SSL layer wants to read, even though the application wants to write. Your job is to do the read on the SSL layer's behalf, and then retry the write. Where "some" is obtained by queried bio2 for the amount of pending data. You can loop reading/sending smaller amounts that if the entire amount is too big to write to the target in one go without blocking. Otherwise, you need a large enough write buffer downstream. If you can't fully drain bio2's output in one go, just keep the socket selected for write and try again later when more space is available. > put them into "bio2" > (thus internally fulfilling bio1's need for read), and then again try > bio_write() to "bio1" WITH IDENTICAL APP-PAYLOAD-BYTES. If it again > returns SSL_ERROR_WANT_READ, we repeat the cycle. Well, you may not fulfill that need, if you don't drain all the pending data, and so the SSL_ERROR_WANT_... may repeat. What you do with bio2 has no direct relationship on the direction of the I/O in bio1. Just keep track of whether the crypto wants to read or write, and whether either bio2 or any buffer you interpose downstream has pending data. Then move data between the network and bio2 as appropriate. > For reading app-payload-bytes, we do a bio_read() from "bio1", and if > "bio1" requires writing bytes to bio2/peer, bio_read() will return > SSL_ERROR_WANT_WRITE (even for blocking sockets). No, it will *usually* return SSL_ERROR_WANT_READ, that's when you read "some" (based on desired read size from bio2) from the network and write the results to bio2. Those reads might be satisfied from a buffer you interpose. However, it might sometimes return SSL_ERROR_WANT_WRITE, in which case, the SSL layer wants to write, even though the application wants to read. Your job is to do the write on the SSL layer's behalf, and then retry the read. > We then pick up the > app-payload-encrypted-bytes from "bio2", transfer them over the wire > via device-specific-socket (thus fulfilling bio1's need to send bytes > to peer), and then again try bio_read() from "bio1" WITH IDENTICAL > APP-PAYLOAD-BYTES. If it again returns SSL_ERROR_WANT_WRITE, we repeat > the cycle. When retrying app-layer reads, you do not need to provide the same buffer or read request size on subsequent attempts. > Obviously, for the above two sub-plots, the only chance of failure if > reading/writing bytes fails over the wire. > But that is not the norm; if it does happen, then everything will be > restarted from scratch (at least by me). > If yes, then it seems everything can be done fully synchronously for > blocking-sockets, No, synchronous operation will not work. The network may not be ready to receive more data when you're ready to write (network layer data), or have more data when you're ready to read. Use of bio-pairs with blocking sockets is highly risky, you'll probably dead-lock with some regularity. > thus eliminating the need of multiple threads, You don't need threads, an event loop will do. Indeed an event loop is generally better, don't know whether the same bio-pair is safe for use by multiple threads. You just need to be able to perform the equivalent of select()/poll()/... on whatever interface you have for moving data to/from the network. -- Viktor.
