Re: ext3 + dm_crypt performance impact (CentOS 5.4 AMD64)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello Milan, 

> > I do this for plain ext3 filesystem on /dev/sdf and ext3 on crypto 
> > filesystem (I use dm_crypt and truecrypt as backends).
> 
> Please note that truecrypt uses dmcrypt in recent versions 
> (and if kernel support requested mode).
> 
> (Note that CentOS 5 kernel cannot do XTS mode, one day it 
> will be backported though.)

Seems like it. I tested with a recent 2.6.32 Kernel and here truecrypt
somehow uses dm_crypt in the backend (in xts mode). 

What I also found was, that doing a simple

  "dmsetup table --showkeys" actually shows the dm_crypt master key in
hex for the disk

Isn't that a little bit too easy ? Should dmsetup not at least scrumble
it (xxxxx) ?

Otherwise this information can easily leak out into ticketing systems,
support attachents etc. Of course you need root to do dmsetup, and root
also has access to the key (henn-egg-problem) so it is not a security
problem. 

> > I also see that the disk writes are split in 4k:
> 
> > Because this is cached I/O (buffer cache), dm_requests are 
> processed 
> > in units of 4k (somehow this seems to be a implementation specific 
> > thing).
> 
> dmcrypt doesn't limit io operation if directly submitted to 
> it, but stacking of devices can limit it. No idea about that 
> ext3 thing though.
> 
> Because in RHEL5 kernel is not yet implemented merge callback 
> in DM targets, all stacked targets must split io operation to 
> 4k (page) units.
> (This is fixed long time ago upstream.)

Tested on 2.6.32 - same results. 
   
   Ext3: only 4k writes and -30% Performance
   XFS:  mixed writes and -15% Performance with Encryption (after
disabeling barrier support, otherwise performance is horrible, backend
should have BBU :) - Amazon EC2)

> > It also looks like dm_crypt does only scale to one CPU Core 
> per device.
> Yes, this is known limitation.
> 
> I know about some problems, some of them are partially fixed 
> in devel tree (and targeted to 2.6.38) some will be based on 
> top of these changes (latency problem is one of them probably).
> 
> I do not want to speculate what exactly causes latency issues 
> - it is not one thing (testing of per-cpu worker threads 
> changes shows that page cache and fs are also important part 
> which can contribute to some problems here etc.)
> 
> But all these change will be upstream - old kernels, like 
> RHEL5 are maintained with focus to stability (most of changes 
> are not backportable anyway).
> 
> If you are Red Hat customer, you can of course request this 
> through support.
> 
> Milan

So after all a little "high level conclusion". 

Dm_crypt adds latency ...

  ... (a) by design (every bit must go throught the cpu) and 
  ... (b) by Implementation (there seem to be currently unsolved
implementation issues, that can increase the latency)

(a) can probably not be fixed (btw: Any estimated on the latency impact
of AES-NI ?? If performance is 20x faster, is latency 20x smaller or
does AES-NI get better performance by parallellization ?)
(b) can be fixed and will not arrive before 2.6.38

Besides that the impact on workload can be described as: 

If having a single disk and working in an area where one cpu core is NOT
loaded a 100% (and thus the solution still scales, probably < 80 MB/s),
the worst case overhead with encryption is somewhere abound 30-40% for
heavily cached workloads. In Database Workloads, where the disk backend
is limiting factor, the impact can be ignored. Here dm_crypt adds CPU
load, scaling with the amount of data encrypted. For SSD's, dm_crypt can
be a limiting factor because of the increased latency. 

Does that match the observations of others ? 

Regards, 
Robert 
_______________________________________________
dm-crypt mailing list
dm-crypt@xxxxxxxx
http://www.saout.de/mailman/listinfo/dm-crypt


[Index of Archives]     [Device Mapper Devel]     [Fedora Desktop]     [ATA RAID]     [Fedora Marketing]     [Fedora Packaging]     [Fedora SELinux]     [Yosemite News]     [KDE Users]     [Fedora Tools]     [Fedora Docs]

  Powered by Linux