Hi Jeremy, have you tried to reproduce with all performance options disabled? They are possibly no good idea on a local system. What local fs do you use? -- Regards, Stephan On Tue, 23 Mar 2010 19:11:28 -0500 Jeremy Enos <jenos at ncsa.uiuc.edu> wrote: > Stephan is correct- I primarily did this test to show a demonstrable > overhead example that I'm trying to eliminate. It's pronounced enough > that it can be seen on a single disk / single node configuration, which > is good in a way (so anyone can easily repro). > > My distributed/clustered solution would be ideal if it were fast enough > for small block i/o as well as large block- I was hoping that single > node systems would achieve that, hence the single node test. Because > the single node test performed poorly, I eventually reduced down to > single disk to see if it could still be seen, and it clearly can be. > Perhaps it's something in my configuration? I've pasted my config files > below. > thx- > > Jeremy > > ######################glusterfsd.vol###################### > volume posix > type storage/posix > option directory /export > end-volume > > volume locks > type features/locks > subvolumes posix > end-volume > > volume disk > type performance/io-threads > option thread-count 4 > subvolumes locks > end-volume > > volume server-ib > type protocol/server > option transport-type ib-verbs/server > option auth.addr.disk.allow * > subvolumes disk > end-volume > > volume server-tcp > type protocol/server > option transport-type tcp/server > option auth.addr.disk.allow * > subvolumes disk > end-volume > > ######################ghome.vol###################### > > #-----------IB remotes------------------ > volume ghome > type protocol/client > option transport-type ib-verbs/client > # option transport-type tcp/client > option remote-host acfs > option remote-subvolume raid > end-volume > > #------------Performance Options------------------- > > volume readahead > type performance/read-ahead > option page-count 4 # 2 is default option > option force-atime-update off # default is off > subvolumes ghome > end-volume > > volume writebehind > type performance/write-behind > option cache-size 1MB > subvolumes readahead > end-volume > > volume cache > type performance/io-cache > option cache-size 1GB > subvolumes writebehind > end-volume > > ######################END###################### > > > > On 3/23/2010 6:02 AM, Stephan von Krawczynski wrote: > > On Tue, 23 Mar 2010 02:59:35 -0600 (CST) > > "Tejas N. Bhise"<tejas at gluster.com> wrote: > > > > > >> Out of curiosity, if you want to do stuff only on one machine, > >> why do you want to use a distributed, multi node, clustered, > >> file system ? > >> > > Because what he does is a very good way to show the overhead produced only by > > glusterfs and nothing else (i.e. no network involved). > > A pretty relevant test scenario I would say. > > > > -- > > Regards, > > Stephan > > > > > > > >> Am I missing something here ? > >> > >> Regards, > >> Tejas. > >> > >> ----- Original Message ----- > >> From: "Jeremy Enos"<jenos at ncsa.uiuc.edu> > >> To: gluster-users at gluster.org > >> Sent: Tuesday, March 23, 2010 2:07:06 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi > >> Subject: gluster local vs local = gluster x4 slower > >> > >> This test is pretty easy to replicate anywhere- only takes 1 disk, one > >> machine, one tarball. Untarring to local disk directly vs thru gluster > >> is about 4.5x faster. At first I thought this may be due to a slow host > >> (Opteron 2.4ghz). But it's not- same configuration, on a much faster > >> machine (dual 3.33ghz Xeon) yields the performance below. > >> > >> ####THIS TEST WAS TO A LOCAL DISK THRU GLUSTER#### > >> [root at ac33 jenos]# time tar xzf > >> /scratch/jenos/intel/l_cproc_p_11.1.064_intel64.tgz > >> > >> real 0m41.290s > >> user 0m14.246s > >> sys 0m2.957s > >> > >> ####THIS TEST WAS TO A LOCAL DISK (BYPASS GLUSTER)#### > >> [root at ac33 jenos]# cd /export/jenos/ > >> [root at ac33 jenos]# time tar xzf > >> /scratch/jenos/intel/l_cproc_p_11.1.064_intel64.tgz > >> > >> real 0m8.983s > >> user 0m6.857s > >> sys 0m1.844s > >> > >> ####THESE ARE TEST FILE DETAILS#### > >> [root at ac33 jenos]# tar tzvf > >> /scratch/jenos/intel/l_cproc_p_11.1.064_intel64.tgz |wc -l > >> 109 > >> [root at ac33 jenos]# ls -l > >> /scratch/jenos/intel/l_cproc_p_11.1.064_intel64.tgz > >> -rw-r--r-- 1 jenos ac 804385203 2010-02-07 06:32 > >> /scratch/jenos/intel/l_cproc_p_11.1.064_intel64.tgz > >> [root at ac33 jenos]# > >> > >> These are the relevant performance options I'm using in my .vol file: > >> > >> #------------Performance Options------------------- > >> > >> volume readahead > >> type performance/read-ahead > >> option page-count 4 # 2 is default option > >> option force-atime-update off # default is off > >> subvolumes ghome > >> end-volume > >> > >> volume writebehind > >> type performance/write-behind > >> option cache-size 1MB > >> subvolumes readahead > >> end-volume > >> > >> volume cache > >> type performance/io-cache > >> option cache-size 1GB > >> subvolumes writebehind > >> end-volume > >> > >> What can I do to improve gluster's performance? > >> > >> Jeremy > >> > >> _______________________________________________ > >> Gluster-users mailing list > >> Gluster-users at gluster.org > >> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users > >> _______________________________________________ > >> Gluster-users mailing list > >> Gluster-users at gluster.org > >> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users > >> > >> > > > > >
