I found strange behavior with this script. You may recall my setup
has a three disk RAID5, kernel 2.6.8, lvm2.
While I can achieve 80MB/s reading from /dev/md1 (my RAID5 device), I
can't get better than 60MB/s from any of the logical volumes which
exist on that array. (/dev/md1 is the only PV in that VG.)
Furthermore, the readahead settings on /dev/md1 don't seem to make any
difference, only the readahead setting on /dev/vg0/lvol0 (for example)
matters. This doesn't make any sense to me. I didn't think LVM was
supposed to impose any significant overhead.
Is the LVM2/DM layer doing its own readahead through to the PV,
regardless of the settings I'm doing with blockdev --setra?
Finally, when trying to test throughput reading from an actual file on
a filesystem, I couldn't figure out how to flush the cache reliably.
"blockdev --flushbufs" works great when the test is reading straight
from the block device but has no effect when reading from a file on a
filesystem. Any advice here? A pointer to an up-to-date in-depth
description (for 2.6) of how the whole cache/buffer thing works would
be very much appreciated.
Thanks!
-Steve
On Sun, 12 Dec 2004 08:56:34 +0000, David Greaves wrote:
> I hacked up a quick script to test permutations of readahead - it's not
> exactly bonnie+++ but it may be useful.
> I wish I'd bothered with mdadm stripe sizes too - but the array is
> pretty full now and I'll live with what it delivers.
>
> Essentially I found the best performance on *my* system with all low
> level devices and the md device set to a 0 readahead and the lvm device
> set to 4096.
> I'm only interested in video streaming big (1+Gb) files. Your needs (and
> hence test) may differ.
>
> my system is 2.6.10-rc2, xfs, lvm2, raid5, sata disks.
>
> cc'ed the lvm group since this often seems to come up in conjunction
> with you guys :)
>
> For your entertainment...
>
> #!/bin/bash
> RAW_DEVS="/dev/sda /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/hdb"
> MD_DEVS=/dev/md0
> LV_DEVS=/dev/huge_vg/huge_lv
>
> LV_RAS="0 128 256 1024 4096 8192"
> MD_RAS="0 128 256 1024 4096 8192"
> RAW_RAS="0 128 256 1024 4096 8192"
>
> function show_ra()
> {
> for i in $RAW_DEVS $MD_DEVS $LV_DEVS
> do echo -n "$i `blockdev --getra $i` :: "
> done
> echo
> }
>
> function set_ra()
> {
> RA=$1
> shift
> for dev in $@
> do
> blockdev --setra $RA $dev
> done
> }
>
> function show_performance()
> {
> COUNT=4000000
> dd if=/dev/huge_vg/huge_lv of=/dev/null count=$COUNT 2>&1 | grep seconds
> }
>
> for RAW_RA in $RAW_RAS
> do
> set_ra $RAW_RA $RAW_DEVS
> for MD_RA in $MD_RAS
> do
> set_ra $MD_RA $MD_DEVS
> for LV_RA in $LV_RAS
> do
> set_ra $LV_RA $LV_DEVS
> show_ra
> show_performance
> done
> done
> done
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at http://vger.kernel.org/majordomo-info.html
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
