Chris, what is the client config? only the server config is attached.. the read-ahead is not really tuned well for gig/e. try smaller page-count (like 2) with a page size of 128kB. You could even try without read-ahead. 2007/11/16, Chris Johnson <johnson@xxxxxxxxxxxxxxxxxxx>: > > On Fri, 16 Nov 2007, Anand Avati wrote: > > Very simple iozone call > > iozone -aN -r 32k -s 131072k > > Now what I probably should be doing is testing this against the actual > load, the real image processing code to be used. Did look at io > caching. That would possibly help if a number of jobs on the cluster > were to crunch the same file at the same time. If that were not the > case I'm not sure what io caching would give me. > > The config? You can't get much simpler than this, maybe by > pulling the read-ahead and write-behind which weren't there > originally. > > volume brick1 > type storage/posix > option directory /home/sdm1 > end-volume > > volume server > type protocol/server > subvolumes brick1 > option transport-type tcp/server # For TCP/IP transport > # option client-volume-filename /etc/glusterfs/glusterfs-client.vol > option auth.ip.brick1.allow * > end-volume > > volume writebehind > type performance/write-behind > option aggregate-size 131072 # in bytes > subvolumes brick1 > end-volume > > volume readahead > type performance/read-ahead > option page-size 65536 ### in bytes > option page-count 16 ### memory cache size is page-count x page-size > per file > subvolumes brick1 > end-volume > > >> > >> Performance is still way below NFS. I have turned on > >> write-behind and read-ahead. And with fuse it's a little faster. but > >> not significantly. > > > > > > What is your IO pattern? what are the tests you are running? what is > your > > interconnect? NFS can provide really good re-read performance which you > can > > get out of glusterfs using io-cache on the client side. > > > > Can we have a discussion on whether I'm heading in the right > >> direction and what order things go in for the config files? > > > > > > Can you paste your config files on pastebin and link it and also tell us > the > > kind of application/IOtests you are running? > > > > thanks, > > avati > > > > > > -- > > It always takes longer than you expect, even when you take into account > > Hofstadter's Law. > > > > -- Hofstadter's Law > > > > > ------------------------------------------------------------------------------- > Chris Johnson |Internet: johnson@xxxxxxxxxxxxxxxxxxx > Systems Administrator |Web: > http://www.nmr.mgh.harvard.edu/~johnson > NMR Center |Voice: 617.726.0949 > Mass. General Hospital |FAX: 617.726.7422 > 149 (2301) 13th Street |Do not meddle in the affairs of wizards, for > Charlestown, MA., 02129 USA |you are crunchy and good with ketchup. > > ------------------------------------------------------------------------------- > -- It always takes longer than you expect, even when you take into account Hofstadter's Law. -- Hofstadter's Law
