On Wed, Mar 30, 2011 at 12:12:14PM -0400, Joe Landman wrote: > > >- T610: > > > >/dev/mapper/cmd1 on / type xfs (rw,inode64,delaylog,logbsize=262144) > > > >- PE2900: > > > >/dev/mapper/cmd1 on / type xfs (rw,inode64,delaylog,logbsize=262144) > > Hmmm. You are layering an LVM atop the raid? Your raids are > /dev/md1. How is /dev/mapper/cmd1 related to /dev/md1? It's a dm-crypt (luks) layer. No lvm here. > [...] > > >>[root@vault t]# dd if=/dev/md2 of=/dev/null bs=32k count=32000 > > > >- T610: > > > >32000+0 enregistrements lus > >32000+0 enregistrements écrits > >1048576000 octets (1,0 GB) copiés, 1,70421 s, 615 MB/s > > > >- PE2900: > > > >32000+0 records in > >32000+0 records out > >1048576000 bytes (1.0 GB) copied, 2.02322 s, 518 MB/s > > Raw reads from the MD device. For completeness, you should also do > > dd if=/dev/mapper/cmd1 of=/dev/null bs=32k count=32000 - T610: 32000+0 records in 32000+0 records out 1048576000 bytes (1.0 GB) copied, 0.21472 s, 4.9 GB/s - PE2900: 32000+0 records in 32000+0 records out 1048576000 bytes (1.0 GB) copied, 0.396733 s, 2.6 GB/s > and > > dd if=/backup/t/big.file of=/dev/null bs=32k count=32000 > > to see if there is a sudden loss of performance at some level. - T610: 32000+0 records in 32000+0 records out 1048576000 bytes (1.0 GB) copied, 0.251609 s, 4.2 GB/s - PE2900: 32000+0 records in 32000+0 records out 1048576000 bytes (1.0 GB) copied, 1.70794 s, 614 MB/s > >>[root@vault t]# dd if=/dev/zero of=/backup/t/big.file bs=32k count=32000 > > > >- T610: > > > >32000+0 enregistrements lus > >32000+0 enregistrements écrits > >1048576000 octets (1,0 GB) copiés, 0,870001 s, 1,2 GB/s > > > >- PE2900: > > > >32000+0 records in > >32000+0 records out > >1048576000 bytes (1.0 GB) copied, 9.11934 s, 115 MB/s > Ahhh ... look at that. Cached write is very different between the > two. An order of magnitude. You could also try a direct > (noncached) write, using oflag=direct at the end of the line. This > could be useful, though direct IO isn't terribly fast on MD raids. - T610: 32000+0 records in 32000+0 records out 1048576000 bytes (1.0 GB) copied, 316.461 s, 3.3 MB/s - PE2900: 32000+0 records in 32000+0 records out 1048576000 bytes (1.0 GB) copied, 262.569 s, 4.0 MB/s > If we can get the other dd's indicated, we might have a better sense > of which layer is causing the issue. It might not be MD. Thanks for your help, -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
