On Fri, Nov 22, 2013 at 17:29:57 CET, Adam Boyhan wrote: > Doing allot of testing with dm-crypt and mdadm. Running into an issues > which is killing me. We run a standard CentOS6 load with kernel > "2.6.32-358.23.2.el6.x86_64" > > Scenario #1: mdadm raid10 -> dm-crypt -> ext4 > Scenario #2: dm-crypt -> mdadm raid10 -> ext4 > > Scenario #1 has no issues and runs reliably but takes a significant hit in > performance due to the single threaded nature of kryptd. I read quite a > bit about people instead encrypting each block device then putting > software encryption on that. This all goes well until I start to > benchmark. The writing process of the benchmark goes well, as soon as I > hit the read portion of the test, the machine panics and locks up. I > initially tested this idea with raid0 and I don't have this panic, it > seems that only raid10 causes the panic. At this point I am stumped as to > what I can do. I am feel this is more an issue with mdadm than dm-crypt > but I figured it was worth getting others opinions. I appreciate any or > all help, if I am missing any details let me know and I will provide them. I have Scenario #1 running for years with a 3-way RAID1 (notebook drives give me about 1 firmware crash/year, hence the 3-way...) I do not have a performance hit. Writing runs at a sustained speed of ~75MB/s with 3 "kworker"s at about 15% CPU each or 1 kworker at 45% (it varies between the two states). Reading is similar. CPU is a Phenom II 945 4-core. That speed is about what the disks can handle with ext3, non-encrypted gives about the same speed. (Not using ext4, still too new for me.) Kernel is a self-compiled stock kernel.org kernel 3.10.17. I would suggest that 2.6.32 is at best called "historic" and entirely unsuitable for any kind of performance-critical set-up. I also strongly suspect (from some evaluation work on CentOS I did recently for a customer) that Red-Hat has messed up updating their own kernel and you may do better with a self-compiled 2.6.32.61. The real benchmark would be with something like 3.10.20 or the like, or maybe 3.12.1. (Careful: In my setup, 3.10.19 has a networking issue, but 3.10.17 has been running stable for some time, so I would recommend that one.) My bottom line is that a) I do not trust the CentOS "stable" kernel one bit. b) 2.6.32 is so old, doing anything on it to improve performance is probably a waste of time. 2.6.32 is 4 years old by now and the 2.6.x line is 10 years old... Arno -- Arno Wagner, Dr. sc. techn., Dipl. Inform., Email: arno@xxxxxxxxxxx GnuPG: ID: CB5D9718 FP: 12D6 C03B 1B30 33BB 13CF B774 E35C 5FA1 CB5D 9718 ---- There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult. --Tony Hoare _______________________________________________ dm-crypt mailing list dm-crypt@xxxxxxxx http://www.saout.de/mailman/listinfo/dm-crypt
