Hello Milan, > > I do this for plain ext3 filesystem on /dev/sdf and ext3 on crypto > > filesystem (I use dm_crypt and truecrypt as backends). > > Please note that truecrypt uses dmcrypt in recent versions > (and if kernel support requested mode). > > (Note that CentOS 5 kernel cannot do XTS mode, one day it > will be backported though.) Seems like it. I tested with a recent 2.6.32 Kernel and here truecrypt somehow uses dm_crypt in the backend (in xts mode). What I also found was, that doing a simple "dmsetup table --showkeys" actually shows the dm_crypt master key in hex for the disk Isn't that a little bit too easy ? Should dmsetup not at least scrumble it (xxxxx) ? Otherwise this information can easily leak out into ticketing systems, support attachents etc. Of course you need root to do dmsetup, and root also has access to the key (henn-egg-problem) so it is not a security problem. > > I also see that the disk writes are split in 4k: > > > Because this is cached I/O (buffer cache), dm_requests are > processed > > in units of 4k (somehow this seems to be a implementation specific > > thing). > > dmcrypt doesn't limit io operation if directly submitted to > it, but stacking of devices can limit it. No idea about that > ext3 thing though. > > Because in RHEL5 kernel is not yet implemented merge callback > in DM targets, all stacked targets must split io operation to > 4k (page) units. > (This is fixed long time ago upstream.) Tested on 2.6.32 - same results. Ext3: only 4k writes and -30% Performance XFS: mixed writes and -15% Performance with Encryption (after disabeling barrier support, otherwise performance is horrible, backend should have BBU :) - Amazon EC2) > > It also looks like dm_crypt does only scale to one CPU Core > per device. > Yes, this is known limitation. > > I know about some problems, some of them are partially fixed > in devel tree (and targeted to 2.6.38) some will be based on > top of these changes (latency problem is one of them probably). > > I do not want to speculate what exactly causes latency issues > - it is not one thing (testing of per-cpu worker threads > changes shows that page cache and fs are also important part > which can contribute to some problems here etc.) > > But all these change will be upstream - old kernels, like > RHEL5 are maintained with focus to stability (most of changes > are not backportable anyway). > > If you are Red Hat customer, you can of course request this > through support. > > Milan So after all a little "high level conclusion". Dm_crypt adds latency ... ... (a) by design (every bit must go throught the cpu) and ... (b) by Implementation (there seem to be currently unsolved implementation issues, that can increase the latency) (a) can probably not be fixed (btw: Any estimated on the latency impact of AES-NI ?? If performance is 20x faster, is latency 20x smaller or does AES-NI get better performance by parallellization ?) (b) can be fixed and will not arrive before 2.6.38 Besides that the impact on workload can be described as: If having a single disk and working in an area where one cpu core is NOT loaded a 100% (and thus the solution still scales, probably < 80 MB/s), the worst case overhead with encryption is somewhere abound 30-40% for heavily cached workloads. In Database Workloads, where the disk backend is limiting factor, the impact can be ignored. Here dm_crypt adds CPU load, scaling with the amount of data encrypted. For SSD's, dm_crypt can be a limiting factor because of the increased latency. Does that match the observations of others ? Regards, Robert _______________________________________________ dm-crypt mailing list dm-crypt@xxxxxxxx http://www.saout.de/mailman/listinfo/dm-crypt
