On Wed, Dec 31, 2008 at 02:42:27PM -0600, Mike McGrath wrote: > Lets pool some knowledge together because at this point, I'm missing > something. > > I've been doing all measurements with sar as bonnie, etc, causes builds to > timeout. > > Problem: We're seeing slower then normal disk IO. At least I think we > are. This is a PERC5/E and MD1000 array. > > When I try to do a normal copy "cp -adv /mnt/koji/packages /tmp/" I get > around 4-6MBytes/s That's sucky. > When I do a cp of a large file "cp /mnt/koji/out /tmp/" I get > 30-40MBytes/s. > > If I "cat /dev/sde > /dev/null" I get between 225-300MBytes/s read. That's about what I would expect for straight block reads. > The above tests are pretty consistent. /dev/sde is a raid5 array, > hardware raid. Remember, RAID 5's worst performance is for writes. In your 14-drive array, it has to calculate parity across all the drives, then write the data across all the drives. As long as it's pure writes (e.g. not read/modify/write) it's not so bad, but still slower than you might think. What ext3 journaling options are enabled (e.g. what does 'mount' say)? If it's data=ordered (the default), that's OK. If it's data=journal, then all the data gets written twice (first to the journal, then the journal to the disk), which is really really slow, and the size of the journal would really make a difference too. RAID controllers also tend to benefit from using the noop scheduler, which effectively defers the scheduling to the RAID controller. Note that cp doesn't fdatasync(), so the I/Os will be scheduled, but not necessarily completed, when cp returns. Which might make your numbers even more optimistic than they really are. :-( -- Matt Domsch Linux Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux _______________________________________________ Fedora-infrastructure-list mailing list Fedora-infrastructure-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
