You might want to wait until 2.0.5 as there is a ton of bug fixes to booster in that release. Either way please let us know how it goes. ls On Jul 30, 2009, at 12:39 AM, Somsak Sriprayoonsakul <somsaks at gmail.com> wrote: > Thank you very much for you reply > > At the time we used 2.0.3, and yes we used stock Apache from CentOS. > I will try 2.0.4 very soon to see if it's work. > > For Booster, it seems not working correctly for me. Booster > complains a lots of error with plain 'ls' command (but giving the > correct output). Also, with booster, Apache process refuse to start. > I will try 2.0.4 to see if it improves. If not, I will attach error > log next time. > > > 2009/7/30 Raghavendra G <raghavendra at gluster.com> > Hi Somsak, > > Sorry for the delayed reply. Below you've mentioned that you've > problems with apache and booster. Going forward, Apache over booster > will be the preferred approach. Can you tell us what version of > glusterfs you are using? And as I can understand you are using > apache 2.2, am I correct? > > regards, > ----- Original Message ----- > From: "Liam Slusser" <lslusser at gmail.com> > To: "Somsak Sriprayoonsakul" <somsaks at gmail.com> > Cc: gluster-users at gluster.org > Sent: Saturday, July 25, 2009 3:46:14 AM GMT +04:00 Abu Dhabi / Muscat > Subject: Re: Gluster 2.0.3 + Apache on CentOS5 > performance issue > > I haven't tried an apples to apples comparison with Apache > +mod_gluster vs > Apache+fuse+gluster however i do run both setups. I load tested > both setups > so to verified it could handle 4x our normal daily load and left it > at that. > I didn't actually compare the two (although that might be cool to do > someday). > I really like the idea of Apache+mod_gluster as I don't have to deal > with > the whole fuse and mounting the filesystem. It always scares me > having a > public facing webserver with your whole backend fileshare mounted > locally. > Its very slick for serving content such as media files. We serve > audio > content to our CDN with a pair of Apache/mod_gluster servers - pushing > 200-300mbit on average daily and everything works very well. > > We run an apache+fuse+gluster setup because we need to run some > mod_perl > before serving the actual content. However performance is still > very good. > We do around 50-100 requests (all jpeg images) per second off of a > fuse > mount and everything works great. We also have a java tomcat+fuse > +gluster > service which does image manipulation on the fly off of a gluster > mount. > > We have two backend gluster servers using replication which serve > all this > content. > > If you would like more information on our setup id be happy to share > offline. Just email me privately. > > thanks, > liam > > On Fri, Jul 24, 2009 at 8:08 AM, Somsak Sriprayoonsakul > <somsaks at gmail.com>wrote: > > > Oh thank you, thought noone will reply me :) > > > > Have you tried Apache + Fuse over GlusterFS? How is the performance? > > > > Also, anyone in this mailing-list have tried Apache with booster? > I tried > > it but Apache refuse to start (just hang and freeze). > > > > 2009/7/23 Liam Slusser <lslusser at gmail.com> > > > > > >> We use mod_gluster and Apache > >> 2.2 with good results. We also ran into the same issue as you > that we ran out of memory past 150 threads even on a 8gig machine. > We got around this by compiling Apache using mpm-worker > >> (threads) instead of prefork - it uses 1/4 as much ram with the > same number > >> of connections (150-200) and everything has been running > smoothly. I cannot > >> see any performance difference except it using way less memory. > >> liam > >> > >> > >> On Sun, Jul 12, 2009 at 5:11 AM, Somsak Sriprayoonsakul < > >> somsaks at gmail.com> wrote: > >> > >>> Hello, > >>> > >>> We have been evaluating the choice for the new platform for a > webboard > >>> system. > >>> The webboard is PHP scripts that generate/modify HTML page when > user > >>> posting/add comment to the page, resulting topic is actually > stored as a > >>> HTML file with all related file (file attach to the topic, > etc.. )stored in > >>> its own directory for each topic. In general, the web site > mostly serve a > >>> lot of small static files using Apache while using PHP to do > other dynamic > >>> contents. This system has been working very well in the past, > with the > >>> increasing page view rate, it is very likely that we will need > some kind of > >>> Cluster file system as backend very soon. > >>> > >>> We have set up a test system using Grinder as stress test tool. > The test > >>> system is 11 machines of Intel Dual Core x86_64 CentOS5 with > stock Apache > >>> (prefork, since the goal is to use this with PHP), linked > together with > >>> Gigabit Ethernet. We try to compare the performance of either > using single > >>> NFS server in sync mode against using 4 Gluster nodes > (distribute of 2 > >>> replicated nodes) through Fuse. However, the transaction per > second (TPS) > >>> result is not good. > >>> > >>> NFS (single server, sync mode) > >>> - 100 thread of client - Peak TPS = 1716.67, Avg. TPS = 1066, > mean > >>> response time = 61.63 ms > >>> - 200 threads - Peak TPS = 2790, Avg. TPS = 1716, mean rt = > 87.33 ms > >>> - 400 threads - Peak TPS = 3810, Avg. TPS = 1800, mean rt = 165ms > >>> - 600 threads - Peak TPS = 4506.67, Avg. TPS = 1676.67, mean rt = > >>> 287.33ms > >>> > >>> 4 nodes Gluster (2 distribute of replicated 2 node) > >>> - 100 thread - peak TPS = 1293.33, Avg. TPS = 430, mean rt = > 207.33ms > >>> - 200 threads - Peak TPS = 974.67, Avg. TPS = 245.33, mean rt = > 672.67ms > >>> - 300 threads - Peak TPS = 861.33, Avg. TPS = 210, mean rt = > 931.33 > >>> (no 400-600 threads since we run out of client machine, sorry). > >>> > >>> gfsd is configured to use 32 thread of iothread as brick. gfs- > client is > >>> configured to use io-cache->write-behind->readahead->distribute- > >replicate. > >>> io-cache cache-size is 256MB. I used patched Fuse downloaded > from Gluster > >>> web-site (build through DKMS). > >>> > >>> As the result yield, it seems that Gluster performance worse with > >>> increasing no. of client. One observation is that the glusterfs > process on > >>> client is taking about 100% of CPU during all the tests. > glusterfsd is > >>> utilizing only 70-80% of CPUs during the test time. Note that > system is Dual > >>> core. > >>> > >>> I also tried using modglusterfs and not using fuse at all to > serve all > >>> the static files and conduct another test with Grinder. The > result is about > >>> the same, 1000+ peak TPS with 2-400 avg. TPS. A problem arise in > this test > >>> that each Apache prefork process used more about twice more > memory and we > >>> need to lower number of httpd processes by about half. > >>> > >>> I tried disable EnableMMAP and it didn't help much. Adjusting > readahead, > >>> write behind according to GlusterOptimization page didn't help > much either. > >>> > >>> My question is, there seems to be bottleneck in this setup, but > how can I > >>> track this? Note that, I didn't do any other optimization other > than what > >>> said above. Are there any best practice configuration for using > Apache to > >>> serve a bunch of small static files like this around? > >>> > >>> Regards, > >>> > >>> Somsak > >>> > >>> > >>> > >>> > >>> _______________________________________________ > >>> Gluster-users mailing list > >>> Gluster-users at gluster.org > >>> http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users > >>> > >>> > >> > > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >
