On 12/7/19 11:44 PM, John Stoffel wrote:
"Stuart" == Stuart D Gathman<stuart@xxxxxxxxxxx> writes:
Stuart> On Tue, Oct 29, 2019 at 12:14 PM Daniel Janzon<daniel.janzon@xxxxxxxxxxx> wrote:
I have a server with very high load using four NVMe SSDs and
therefore no HW RAID. Instead I used SW RAID with the mdadm tool.
Using one RAID5 volume does not work well since the driver can only
utilize one CPU core which spikes at 100% and harms performance.
Therefore I created 8 partitions on each disk, and 8 RAID5s across
the four disks.
Now I want to bring them together with LVM. If I do not use a striped
volume I get high performance (in expected magnitude according to disk
specs). But when I use a striped volume, performance drops to a
magnitude below. The reason I am looking for a striped setup is to
Stuart> The mdadm layer already does the striping. So doing it again
Stuart> in the LVM layer completely screws it up. You want plain JBOD
Stuart> (Just a Bunch Of Disks).
Umm... not really. The problem here is more the MD layer not being
able to run RAID5 across multiple cores at the same time, which is why
he split things the way he did.
But we don't know the Kernel version, the LVM version, or the OS
release so as to give better ideas of what to do.
The biggest harm to performance here is really the RAID5, and if you
can instead move to RAID 10 (mirror then stripe across mirrors) then
you should be a performance boost.
As Daniel says, he's got lots of disk load, but plenty of CPU, so the
single thread for RAID5 is a big bottleneck.
Perhaps set "/sys/block/mdx/md/group_thread_cnt" could help here,
see below commits:
commit b721420e8719131896b009b11edbbd27d9b85e98
Author: Shaohua Li <shli@xxxxxxxxxx>
Date: Tue Aug 27 17:50:42 2013 +0800
raid5: sysfs entry to control worker thread number
commit 851c30c9badfc6b294c98e887624bff53644ad21
Author: Shaohua Li <shli@xxxxxxxxxx>
Date: Wed Aug 28 14:30:16 2013 +0800
raid5: offload stripe handle to workqueue
Thanks,
Guoqing
_______________________________________________
linux-lvm mailing list
linux-lvm@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
