Re: Improving performance?

[Markus Krainz <ldm@xxxxxx>]

Hi.

On 2010-11-11 11:49, Lasse Jensen wrote:

Hi. I have a RAID 5 array with 3 (soon upgrading to 4 + hotspare = 5) encrypted drives connected to a system with a Core 2 Duo @ 2.5 ghz  running Debian Squeeze.
Each drive has been formatted with

cryptsetup luksFormat /path/to/device

And put together in a array with

mdamd -C /dev/md0 --raid-level=5 /path/to/first-device /path/to/third-device /path/to/third-device

It works great, and encrypting the devices separately allows me to run more than one instance of kcryptd, thus using both cores in my server. It compensates for the overhead of encrypting the checksumming data seperately, compared to raw devices -> RAID -> encryption and still give me improved speed.

Have you ever done the test and unplugged a drive from your raid and assessed if the raid5 still works?
My experience with this setup is that cryptsetup and mdadm do not work well, if the underlying device suddenly disappears.

First unplug 1 of your 3 drives, look if it still works and mdadm recognises the missing drive using mdadm --detail /dev/raidname
Then reconnect the drive without restarting the computer simulating a new device to replace the old one.
Try if you can still open it with cryptsetup (using the same name).
Try if you can rebuild the array.

Could you please try it and poste the results here?

At the moment, i get 70 mb/s sequential read speed locally. I would like to boost it to at least 100 or even more, as 1) the raw drives support way more and 2) i would like to fill my gigabit ethernet when copying files over the network.

Now, what are my options?

A quadcore CPU like the Q6600 would double the number of cores and theoretically double the throughput, but at cost of idle power. Note that the server is idle most of the time.
A core i5. They have AES support in hardware, but it's an expensive solution and i'm not even sure it has Linux support.
A PCI or PCIe based card, like the HiFN cards, but what card should i look for and what speed should i expect?
Using the CUDA cores of my nVidia card, but no driver seems to exists for that.

The first option is pretty straight forward, but what about the rest? Or are there any other options i havent thought of?

I have a setup with an i5 and another one with q6600. Notice that the q6600 does not fit on the same motherboards as the i5.
dmcrypt/luks is used on top of the raid. The performance of the i5 is not great, despite hardware aes. Should not be this numbers a bit higher than 158 MB/sec?

~/httptunnel-3.3/ hdparm -t --direct /dev/md1           

/dev/md1:
 Timing O_DIRECT disk reads:  936 MB in  3.00 seconds = 311.67 MB/sec

~/httptunnel-3.3/ hdparm -t --direct /dev/mapper/evol

/dev/mapper/evol:
 Timing O_DIRECT disk reads:  476 MB in  3.01 seconds = 158.30 MB/sec

cat /proc/cpuinfo | grep -E (model name|aes)
model name      : Intel(R) Core(TM) i5 CPU         660  @ 3.33GHz
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt aes lahf_lm ida arat dts tpr_shadow vnmi flexpriority ept vpid


Regards,
Markus Krainz

--
Lasse Jensen (fafler at gmail dot com)

_______________________________________________
dm-crypt mailing list
dm-crypt@xxxxxxxx
http://www.saout.de/mailman/listinfo/dm-crypt

_______________________________________________
dm-crypt mailing list
dm-crypt@xxxxxxxx
http://www.saout.de/mailman/listinfo/dm-crypt