On 12/30/2010 08:12 PM, Stuart D. Gathman wrote:
On Thu, 30 Dec 2010, Spelic wrote:
Also there is still the mystery of why times appear *randomly* related to the
number of devices, chunk sizes, and stripe sizes! if the rmw cycle was the
culprit, how come I see:
raid5, 4 devices, 16384k chunk: 41sec (4.9MB/sec)
raid5, 6 device, 4096k chunk: 2m18sec ?!?! (1.44 MB/sec!?)
the first has much larger stripe size of 49152K , the second has 20480K !
Ok, next theory. Pvmove works by allocating a mirror for each
contiguous segment of the source LV, update metadata
Ok never mind, I found the problem:
LVM probably uses O_DIRECT, right?
Well it's absymally slow on MD parity raid (I checked with dd on the
bare MD device just now) and I don't know why it's so slow. It's not
because of the rmw because it's slow even the second time I try, when it
does not read anything anymore because all reads are in cache already.
I understand this is probably to be fixed at MD side (and I will report
the problem to linux-raid, but I see it has already been discussed
without much results)
However...
...is there any chance you might fix it at lvm side too, changing LVM to
use nondirect IO so to "support" MD?
In my raid5 array between direct and nondirect (dd bs=1M or smaller)
there's the difference of 2.1MB/s to 250MB/sec, and would probably be
greater on larger arrays.
Also in raid10 nondirect is much faster for small transfer sizes like
bs=4K (28MB/sec to 160MB/sec) but not at 1M, however LVM probably uses
low transfer sizes, right?
Thank you
_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
