Clemens Fruhwirth wrote: >> In kernel 2.6.24 is only one thread per crypt device processing writes. >> Any chance to test it with 2.6.24? > > Why is this necessary at all? dm-crypt folklore tells that writes are > faster than reads because there is no queuing necessary. At least this > was true up to 2.6.1x. I remember that I have done benchmarks that > showed about 25mb/s read speed and (near native) 50mb/s write > speed. The difference between these two were attributed to the > additional queuing and processing by kcryptd that was necessary for > reads. Hi Clemens, here is some info why these changes: In kernels <=2.6.18 were writes processed directly in dm-crypt target map function, not using workqueues at all. Because of changes in block layer (namely serialization in generic_make_request function) write encryption was moved to workqueue (to prevent possible deadlocks in low-memory situations). (kernels 2.6.19 - 2.6.23) There were per-cpu threads for all work, common for all encrypted devices. In 2.6.24 design changed to 2 single-thread per crypt device. (So if you create new device, there are new private threads for it.) Main reasons for this change: - support asynchronous crypto api - plans to support barriers again on dm-crypt (impossible with previous implementation - it can reorder writes) (- and, there was not too big performance effect with this on some configurations - I saw crypto work mostly periodically moving among cpu[-cores] without real parallel encryption. I didn't analysed reason for this - we had to switch to private threads anyway.) > This design seemed to change for 2.6.24. Now writes seem to go also > via the kcryptd/workqueue route. In addition, there is kcrypt_io that > seems to be handling IO request management in a separate thread (sorry > I'm not familiar with the details here). Note, that in sync crypto mode are reads in fact processed in "io" thread, so there is one crypto thread for read, one for write. > In 2.6.25 async cipher > patches were merged. I wonder how they are supposed to be async if > they are not handled by another thread. This is design of crypto api - if there is asynchronous method available, it submits to some "crypto black box" which asynchronously returns completion. There is no need for another thread in dm-crypt for it. If there is no async method, it crypts the content synchronously, (no change from previous version of dm-crypt). (I think this mode is designed mainly for support of crypto HW, I run current implementation of cryptd in crypto layer for async mode test - it is significantly slower than sync mode now for dm-crypt.) (Maybe better ask Herbert about async crypto plans.) But anyway, as alternative, I can imagine in future using it for parallel processing of crypt request instead of adding additional threads to dm-crypt.) Note that async mode support is new in 2.6.25 and I am expecting some issues. (You have beed warned:-) But if there is no async cipher method available, the speed and stability should be as the same as in 2.6.24. Milan -- mbroz@xxxxxxxxxx > So, from 2.6.1x where we had one thread for reads, we now have 2-3 > threads for read and writes. Intuition should tell you that 2.6.24 > would be slower than the rest. But the opposite is the > case. Benchmarking shows 2.6.24 to be the fastest among all its > predecessors. > > Is this due to some clever queue management in workqueues? Or where > does these improvements come from? --------------------------------------------------------------------- dm-crypt mailing list - http://www.saout.de/misc/dm-crypt/ To unsubscribe, e-mail: dm-crypt-unsubscribe@xxxxxxxx For additional commands, e-mail: dm-crypt-help@xxxxxxxx
