Grigory, patch-631 has some changes in read-ahead which should make read-write work better in the cases you have tried below. thanks, avati 2007/12/19, Grigory Shamov <gas@xxxxxx>: > > Dear GlusterFS developers, > > I'm considering using GlusterFS on our ner parallel (two nodes, 10Gb > Ethernet) centralized fileserver for our HPC clusters (several small > ones, tens of CPUs). So I did performance tests for the latest GlusterFS > as well as plain NFS and the recent Lustre-1.6.3. > > The GlusterFS looks very attractive because I understand that unlike > Lustre, one could use it for non-x86 Linux platforms as well, which we > might have gotted in the near future. > > So I did a Bonnie++ benchmark using one of the servers (Dual Opteron, > 4GB RAM, SATA disk, CentOS Linux 5) and a client (old P4 2.4GHz box, > 512MB RAM, Gigabit Ethernet, CentOS 4.5). I used 8Gb size for the > Bonnie++ tests, and tried either forced flush IO or not (-b option; the > data below are for the latter case). Some of the Bonnie++ results are > like this: > > =============================================== > FileSystem: Sequential Output , K/sec > Per-char Block Rewrite > =============================================== > NFS 14442 30419 7710 > Lustre 16012 35228 19018 > GlusterFS 16582 15833 8358 > GlusterFS, wb 17988 43774 8409 > GlusterFS, ra 18414 15863 1804 > GlusterFS, ra, wb 22403 41821 355 > =============================================== > FileSystem: Sequential Input, K/sec Random > Per-char Block seeks, #/s > =============================================== > NFS 20229 49510 178.8 > Lustre 17284 47753 53.0 > GlusterFS 16791 16815 161.4 > GlusterFS, wb 15304 17438 174.1 > GlusterFS, ra 19420 54803 143.3 > GlusterFS, ra, wb 19900 54427 144.4 > =============================================== > > Without performance translators, GlusterFS was as good as non-buffered > IO/ At the same time, Rewrite and Seek tests were OK (about 8000 K/s > and 170 seeks/s). > > Then I applied read-ahead and write-behind translators on the client > side. Blocked reads and writes reached the same or better level as of > NFS or Lustre; but the Rewrite test of Bonnie++ became much worse (an > order of magnitude, actually, below 800 K/s). And there is no > significant fall in Seek test, so I guess the bad Rewrite results are > related to how the wb and ra translators do write and read, not seek. > > So, could you advice me, whether there is a solution for this -- can I > have it both ways with GlusterFS, good IO bandwidth and fast rewrite? > And if yes, how to tune it? Thank you very much! > > > -- > Best regards, > Grigory Shamov > Kazan Science Centre of RAS, > Kazan, Russian Federation > > > > > _______________________________________________ > Gluster-devel mailing list > Gluster-devel@xxxxxxxxxx > http://lists.nongnu.org/mailman/listinfo/gluster-devel > -- If I traveled to the end of the rainbow As Dame Fortune did intend, Murphy would be there to tell me The pot's at the other end.
