Well, here you go. As an aside, one thing I noted in my limited tests is that
raw network/bus/disk performance makes a huge difference; I was surprised
about some of the differences I observed in NFS performance between diff.
pairs of clients / servers ...
All the commented out stuff reflects things I tried before; the last
number was obtained with the files as shown. In my limited testing,
none of the performance options helped; neither did using the custom
fuse.ko
HTH,
Stefan
---------------------------------
Server:
# the physical data space
##volume brick # watch out
volume gfs
type storage/posix
option directory /data/export
end-volume
## the actual exported volume
#volume gfs
# type performance/io-threads
# option thread-count 8
# option cache-size 64MB
# subvolumes brick
#end-volume
# server declaration
volume server
type protocol/server
subvolumes gfs
option transport-type tcp/server # For TCP/IP transport
option auth.ip.gfs.allow *
end-volume
---------------------------------
client
volume brick1
type protocol/client
option transport-type tcp/client # for TCP/IP transport
option remote-host x.y.z.w # IP address of omega
option remote-subvolume gfs # name of the remote volume on omega
end-volume
#;#volume brick2
#;# type protocol/client
#;# option transport-type tcp/client # for TCP/IP transport
#;# option remote-host x.y.z.v # IP address of sigma
#;# option remote-subvolume gfs # name of the remote volume on sigma
#;#end-volume
#;#
#;#volume afr
#;# type cluster/afr
#;# subvolumes brick1 brick2
#;#end-volume
## performance block for cluster # optional!
#volume writeback
# type performance/write-behind
# option aggregate-size 131072
# subvolumes afr
#end-volume
## performance block for cluster # optional!
#volume readahead
# type performance/read-ahead
# option page-size 65536
# option page-count 16
# subvolumes writeback
#end-volume
On Fri, Sep 19, 2008 at 01:07:08AM +0530, Chandranshu . wrote:
> Hi Stefan,
>
> We have been trying to optimize Glusterfs on our network and our results
> match your previous results. After reading your post, I tried removing AFR
> but our performances are still poor as compared to NFS (particulalrly for
> small files). Could you please post your new volume spec file and/or any
> other improvements that you made to any of the layers? That'll be very
> helpful.
>
> Thanks and regards
> Chandranshu
>
>
> > Date: Thu, 18 Sep 2008 10:47:22 +0200
> > From: stefan at mdy.univie.ac.at (Stefan Boresch)
> > Subject: Re: Poor performance with AFR
> > To: gluster-users at gluster.org
> > Message-ID: <20080918084722.GH9860 at loop.mdy.univie.ac.at>
> > Content-Type: text/plain; charset=us-ascii
> >
> > Dear everyone,
> >
> > thank you for your replies. I have some additional data that have
> > clarified issues for me a bit:
> >
> > I have repeated my tests without AFR (basically replicating the plain
> > NFS setup)
> >
> >
> > > > CP-A MAKE
> > > > local disk < 5sec < 0.3sec
> > > > NFS (100MBit) 55sec+-2sec < 2sec
> > > > glusterfs (I) 4m29sec 17sec
> > > > glusterfs (II) 4m05sec 18sec
> > glusterfs w/o AFR 45+-2 sec 9sec <==NEW
> >
> > So, most of the poor performance is due to AFR. Note that the
> > copy actually is now faster than NFS. Interestingly, make
> > still runs much slower (although compared to the actual
> > compile time, this overhead should be negligible in practice)
> >
> > Also, upon running some NFS benchmarks between the two servers, I
> > noted some strange results, letting me suspect some creeping hardware
> > issues.
> >
> > So, I guess I'll (a) wait for glusterfs 1.4.x and (b) look out for
> > some better hardware to test things in the meantime.
> >
> > Sorry if I caused confusion; I should have checked some of these things
> > earlier.
> >
> > Best regards,
> >
> > Stefan Boresch
> >
> > --
> > Stefan Boresch
> > Institute for Computational Biological Chemistry
> > University of Vienna, Waehringerstr. 17 A-1090 Vienna, Austria
> > Phone: -43-1-427752715 Fax: -43-1-427752790
> >
> >
--
Stefan Boresch
Institute for Computational Biological Chemistry
University of Vienna, Waehringerstr. 17 A-1090 Vienna, Austria
Phone: -43-1-427752715 Fax: -43-1-427752790
