On Tue, Feb 09, 2010 at 12:54:16AM +0100, Jakob Sandgren wrote: > Hi, > > (please keep me on CC since I'm not subscribed yet) > > >> I'm using dm-crypt for several mappings with a hardware raid backend. > >> Using a raw read from the raid device (e.g sda) gives ~250MB/s > >> > >> But when I read from an encrypted mapping, I just get ~70MB/s. That > >> should be fine if I at least have the kcryptd process using a core > >> at > >> 100%, but that is not the case. Three of my four cores is 99% idle > >> and > >> one core is 50% idle (aprox.). > > > >Which means your core is too slow to support the full 250MB/s > >speed. > > > >> I have recently upgraded my hardware from an older quadcore system > >> (AMD) to a new Core I7 (860) and expected improved performance and > >> when I did not get that, then did I do some more investegation and > >> found out above. I have also read posts from others having the same > >> problems, but no explanation. > > > >I suspect as the core can support only about 140MB/s encryption > >speed, the accesses get broken. It is well possible that > >if your array would only give 120MB/s it would still have > >that rate encrypted. > > > This does not make sense to me, I can not understand how a "to fast" > disk could give worse results? Disk requests would get issued at the > speed that decryption can handle(?). I do not understand what a > "broken access" would be. Ok, if you are not too fast, then data can be read uninterrupted with maximum size accesses. If you read too fast, then the disk accesses have to be made smaller and wait for the decryption to finish. That adds waiting times and data is not read full speed anymore. It is not so bad with HDDs, the possibly worst case is tapes: If you process slower than the tape streams, it has to stop and rewind frequently, killing performance. > Anyway, just to try the theory did I set up a single disk that would > give 120MB sustained read from the unencrypted mapping, but when I > read from the encrypted mapping I still ended up with the low 70MB/s > and a lot of idle cpu. Hmm. > Running two reads at the same time (to the same encrypted mapping) > actually increased the combined read rate with ~10% ?! > > To me it seems like there is some serious flaw within kcryptd that > ends up to wait for "something" instead of sending enough requests to > the disks to make sure it has data to decrypt. What do you think? The same thing. Here is a reference test (I have notebook disks in this server): Raw read: 54MB/s 14% CPU Read with decrypt: 53MB/s 65% CPU Another idea: Are these 50% CPU on the faked Intel cores, i.e. 50% of a hyperthreading core? This could actually mean 100% on a proper core. You get twice as many hyperthreading pseudo cores, but they are not full cores and can often not perform at 100% as some infrastructure is shared between two halfes of them. So if one half runs at 100% and one half at 0% (as the other half needs something available only once at full load), the complete core load could be reported as 50% when in fact it is 100%. That would mean the crypto is pretty slow on your new CPU. As a reference, my 53MB/s at 65% CPU is on an 2800MHz Athlon 64 X2 5600+ with aes-cbc-plain. Here is an OpenSSL crypto speed test: openssl speed -evp aes-256-cbc [...] The 'numbers' are in 1000s of bytes per second processed. type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes aes-256-cbc 71848.00k 98649.49k 110187.78k 113646.25k 114666.15k You might want to compare this with the numbers on your CPU. Arno -- Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@xxxxxxxxxxx GnuPG: ID: 1E25338F FP: 0C30 5782 9D93 F785 E79C 0296 797F 6B50 1E25 338F ---- Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans If it's in the news, don't worry about it. The very definition of "news" is "something that hardly ever happens." -- Bruce Schneier _______________________________________________ dm-crypt mailing list dm-crypt@xxxxxxxx http://www.saout.de/mailman/listinfo/dm-crypt
