On Mon, Feb 16, 2009 at 7:26 PM, Jeff Sturm <jeff.sturm@xxxxxxxxxx> wrote: >> -----Original Message----- >> From: linux-cluster-bounces@xxxxxxxxxx >> [mailto:linux-cluster-bounces@xxxxxxxxxx] On Behalf Of Corey Kovacs >> Sent: Monday, February 16, 2009 1:55 PM >> To: linux-cluster@xxxxxxxxxx >> Subject: More GFS2 tuning... >> >> By my reckoning, I should be able to see 400MB or more >> sustained throughput using this setup. If this is a pipe >> dream, someone let me know quick before I go nutz. > > What do you get from the raw device? (I.e. if you remove GFS/NFS from > the picture.) Haven't tried this yet, will do it in the morning. > >> The bo values start at around 200MB, then drop down to 0 in >> most cases for a few seconds, then spike to ~700MB/s then >> eases back down to 200, 150 and back down to 0. It looks very >> much like a cacheing issue to me. > > Linux virtual memory does some funny things with fs caching. Try some > tests with O_DIRECT to bypass the buffer cache. On RHEL 5 systems, you > can achieve that with "dd ... oflag=direct" and varying block sizes. > >> I've read that GFS2 is supposed to be "self tuning" but I >> don't think these are necessarily GFS2 issues. This is a new one for me, I'll try this in the morning. > Agreed. If you can experiment with the hardware, what do you get from > other fs types? (such as ext3) > These tests are on the roadmap as well. >> Anyone have something similar? What I/O rates are people getting? > > I don't have any FC hardware quite as nice as yours, but multipathing > AoE over a pair of GigE connections we can get 200MB/s raw, sequential > throughput. (I.e. about the limits of the interconnects.) > > My GFS filesystems are mostly a collection of very small (~1MB or less) > files, so, it's hard to say how they are performing. I'm much more > concerned about the rate of file creates over GFS than raw throughput > right now... > > Jeff > > > -- > Linux-cluster mailing list > Linux-cluster@xxxxxxxxxx > https://www.redhat.com/mailman/listinfo/linux-cluster > Thanks -Corey -- Linux-cluster mailing list Linux-cluster@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/linux-cluster
