On Sat, 29 Aug 2009 09:26:11 -0700 "Nehls, Patrick" <pnehls@xxxxxxxx> wrote: > > A few questions: Are you using lvm at all? Have you tested this scenario with raid 0 or 1? How are your hard drives hooked up in the system? > I'm not using LVM. So far I've only used cryptoloop with RAID1, not dm-crypt. As the RAID 1 is my backup, I wanted to diversify my implementations in case some dm-crypt bug eats my data. I'd rather not mess with that now, but I have two more disks that I will deploy on the machine, as soon as I get an SSD for my desktop - then I'll be able to do some testing. As for the hookup, my cryptoloop RAID 1 and root disk are running off 01:06.0 RAID bus controller: Silicon Image, Inc. PCI0680 Ultra ATA-133 Host Controller (rev 02) and 00:09.0 IDE interface: nVidia Corporation MCP65 IDE (rev a1) whereas the RAID 5 disks are attached via 00:0a.0 SATA controller: nVidia Corporation MCP65 AHCI Controller (rev a3) I previously failed to mention that I am using JFS on the dm-crypt and e2 for cryptoloop - this may or may not influence performance due to chunk sizing issues, especially with crypt-chunk-sizing also playing a role. (mutliple reads from the same sector, offset issues...) It might be interesting to check if something like dd if=/dev/mapper/test | user-space_crypto >/dev/null would provide different results -- I just haven't started looking for a suitable user space AES implementation yet. Best Rick > > -----Original Message----- > From: dm-crypt-bounces@xxxxxxxx [mailto:dm-crypt-bounces@xxxxxxxx] On Behalf Of Rick Moritz > Sent: Friday, August 28, 2009 11:03 AM > To: dm-crypt-list > Subject: strange iowait-cycles > > Dear List, > > I have encountered a performance issue with dm-crypt, for which the algorithms at Google could not point me to a solution. > I am running dm-crypt on a RAID 5 md device. The RAID is capable of sustained read rates of around 150MB/s. > Using dm-crypt I am able to get up to slightly more than 20MB/s read performance. > Now, this would not surprise me all that much, but cryptoloop on another RAID (capable of 30-35 MB/s sustained read) is giving me 30MB/s. > > The interesting issue is that during a read from the mapped/crypted device top reports around 20-30% CPU usage by kcryptd, and a whopping 70-80% iowait cycles. Now, I don't quite understand what is summed up under iowait. If this extends to the memory subsystem, I may just be memory/cache starved (sempron 3200: 1800Mhz, 128kB L2 cache, DDR2 533). If it doesn't, then I am completely clueless, as clearly the underlying I/O-subsystem is readily capable of sustaining higher datarates. > > Even worse than reads are writes. I get an initial burst of about 80-100MB (each disk in the 4 disk array has only 16MB cache) at 25 MB/s, then throughput falls to 8MB/s. > > One possibility that I just came up with is that this performance drop may be due to mapper-requests being non-sequential, where raw device requests are perfectly sequential - but nobody else has mentioned similar issues, so this seems strange. > > On to the boring part: > /dev/mapper//dev/mapper/0 is active: > cipher: aes-cbc-plain > keysize: 256 bits > device: /dev/md0 > offset: 0 sectors > size: 2344252416 sectors > mode: read/write > > cryptsetup is at version 1.0.6-r2 according to Portage. > > Linux Skeletor 2.6.25-hardened-r8 #1 Sat Nov 22 23:22:01 CET 2008 i686 AMD Sempron(tm) Processor 3200+ AuthenticAMD GNU/Linux > > > There's a whole lot of PAX and GRSec going on, like adress space randomization, in case that is of interest. > > I'd be hugely thankful if anyone could check performance on low-cache machines and performance versus cryptoloop. > I'm currently not willing to upgrade the kernel, waiting for version 2.6.29 to be declared stable by the Gentoo hardened team. > Any tips would als be appreciated. > > Thanks all! > > Rick _______________________________________________ dm-crypt mailing list dm-crypt@xxxxxxxx http://www.saout.de/mailman/listinfo/dm-crypt
