Hi, On Thu, Jul 30, 2009 at 11:39 AM, Somsak Sriprayoonsakul<somsaks at gmail.com> wrote: > Thank you very much for you reply > > At the time we used 2.0.3, and yes we used stock Apache from CentOS. I will > try 2.0.4 very soon to see if it's work. > > For Booster, it seems not working correctly for me. Booster complains a lots > of error with plain 'ls' command (but giving the correct output). Also, with Can you mail those errors? > booster, Apache process refuse to start. I will try 2.0.4 to see if it > improves. If not, I will attach error log next time. logs are very much appreciated. > > > 2009/7/30 Raghavendra G <raghavendra at gluster.com> >> >> Hi Somsak, >> >> Sorry for the delayed reply. Below you've mentioned that you've problems >> with apache and booster. Going forward, Apache over booster will be the >> preferred approach. Can you tell us what version of glusterfs you are using? >> And as I can understand you are using apache 2.2, am I correct? >> >> regards, >> ----- Original Message ----- >> From: "Liam Slusser" <lslusser at gmail.com> >> To: "Somsak Sriprayoonsakul" <somsaks at gmail.com> >> Cc: gluster-users at gluster.org >> Sent: Saturday, July 25, 2009 3:46:14 AM GMT +04:00 Abu Dhabi / Muscat >> Subject: Re: Gluster 2.0.3 + Apache on CentOS5 performance >> ?issue >> >> I haven't tried an apples to apples comparison with Apache+mod_gluster vs >> Apache+fuse+gluster however i do run both setups. ?I load tested both >> setups >> so to verified it could handle 4x our normal daily load and left it at >> that. >> ?I didn't actually compare the two (although that might be cool to do >> someday). >> I really like the idea of Apache+mod_gluster as I don't have to deal with >> the whole fuse and mounting the filesystem. ?It always scares me having a >> public facing webserver with your whole backend fileshare mounted locally. >> ?Its very slick for serving content such as media files. ?We serve audio >> content to our CDN with a pair of Apache/mod_gluster servers - pushing >> 200-300mbit on average daily and everything works very well. >> >> We run an apache+fuse+gluster setup because we need to run some mod_perl >> before serving the actual content. ?However performance is still very >> good. >> ?We do around 50-100 requests (all jpeg images) per second off of a fuse >> mount and everything works great. ?We also have a java tomcat+fuse+gluster >> service which does image manipulation on the fly off of a gluster mount. >> >> We have two backend gluster servers using replication which serve all this >> content. >> >> If you would like more information on our setup id be happy to share >> offline. ?Just email me privately. >> >> thanks, >> liam >> >> On Fri, Jul 24, 2009 at 8:08 AM, Somsak Sriprayoonsakul >> <somsaks at gmail.com>wrote: >> >> > Oh thank you, thought noone will reply me :) >> > >> > Have you tried Apache + Fuse over GlusterFS? How is the performance? >> > >> > Also, anyone in this mailing-list have tried Apache with booster? I >> > tried >> > it but Apache refuse to start (just hang and freeze). >> > >> > 2009/7/23 Liam Slusser <lslusser at gmail.com> >> > >> > >> >> We use mod_gluster and Apache >> >> 2.2 with good results. ?We also ran into the same issue as you that we >> >> ran out of memory past 150 threads even on a 8gig machine. ?We got around >> >> this by compiling Apache using mpm-worker >> >> (threads) instead of prefork - it uses 1/4 as much ram with the same >> >> number >> >> of connections (150-200) and everything has been running smoothly. ?I >> >> cannot >> >> see any performance difference except it using way less memory. >> >> liam >> >> >> >> >> >> On Sun, Jul 12, 2009 at 5:11 AM, Somsak Sriprayoonsakul < >> >> somsaks at gmail.com> wrote: >> >> >> >>> Hello, >> >>> >> >>> We have been evaluating the choice for the new platform for a webboard >> >>> system. >> >>> The webboard is PHP scripts that generate/modify HTML page when user >> >>> posting/add comment to the page, resulting topic is actually stored as >> >>> a >> >>> HTML file with all related file (file attach to the topic, etc.. >> >>> )stored in >> >>> its own directory for each topic. In general, the web site mostly >> >>> serve a >> >>> lot of small static files using Apache while using PHP to do other >> >>> dynamic >> >>> contents. This system has been working very well in the past, with the >> >>> increasing page view rate, it is very likely that we will need some >> >>> kind of >> >>> Cluster file system as backend very soon. >> >>> >> >>> We have set up a test system using Grinder as stress test tool. The >> >>> test >> >>> system is 11 machines of Intel Dual Core x86_64 CentOS5 with stock >> >>> Apache >> >>> (prefork, since the goal is to use this with PHP), linked together >> >>> with >> >>> Gigabit Ethernet. We try to compare the performance of either using >> >>> single >> >>> NFS server in sync mode against using 4 Gluster nodes (distribute of 2 >> >>> replicated nodes) through Fuse. However, the transaction per second >> >>> (TPS) >> >>> result is not good. >> >>> >> >>> NFS (single server, sync mode) >> >>> ?- 100 thread of client - Peak TPS = 1716.67, Avg. TPS = 1066, mean >> >>> response time = 61.63 ms >> >>> ?- 200 threads - Peak TPS = 2790, Avg. TPS = 1716, mean rt = 87.33 ms >> >>> ?- 400 threads - Peak TPS = 3810, Avg. TPS = 1800, mean rt = 165ms >> >>> ?- 600 threads - Peak TPS = 4506.67, Avg. TPS = 1676.67, mean rt = >> >>> 287.33ms >> >>> >> >>> 4 nodes Gluster (2 distribute of replicated 2 node) >> >>> - 100 thread - peak TPS = 1293.33, Avg. TPS = 430, mean rt = 207.33ms >> >>> - 200 threads - Peak TPS = 974.67, Avg. TPS = 245.33, mean rt = >> >>> 672.67ms >> >>> - 300 threads - Peak TPS = 861.33, Avg. TPS = 210, mean rt = 931.33 >> >>> (no 400-600 threads since we run out of client machine, sorry). >> >>> >> >>> gfsd is configured to use 32 thread of iothread as brick. gfs-client >> >>> is >> >>> configured to use >> >>> io-cache->write-behind->readahead->distribute->replicate. >> >>> io-cache cache-size is 256MB. I used patched Fuse downloaded from >> >>> Gluster >> >>> web-site (build through DKMS). >> >>> >> >>> As the result yield, it seems that Gluster performance worse with >> >>> increasing no. of client. One observation is that the glusterfs >> >>> process on >> >>> client is taking about 100% of CPU during all the tests. glusterfsd is >> >>> utilizing only 70-80% of CPUs during the test time. Note that system >> >>> is Dual >> >>> core. >> >>> >> >>> I also tried using modglusterfs and not using fuse at all to serve all >> >>> the static files and conduct another test with Grinder. The result is >> >>> about >> >>> the same, 1000+ peak TPS with 2-400 avg. TPS. A problem arise in this >> >>> test >> >>> that each Apache prefork process used more about twice more memory and >> >>> we >> >>> need to lower number of httpd processes by about half. >> >>> >> >>> I tried disable EnableMMAP and it didn't help much. Adjusting >> >>> readahead, >> >>> write behind according to GlusterOptimization page didn't help much >> >>> either. >> >>> >> >>> My question is, there seems to be bottleneck in this setup, but how >> >>> can I >> >>> track this? Note that, I didn't do any other optimization other than >> >>> what >> >>> said above. Are there any best practice configuration for using Apache >> >>> to >> >>> serve a bunch of small static files like this around? >> >>> >> >>> Regards, >> >>> >> >>> Somsak >> >>> >> >>> >> >>> >> >>> >> >>> _______________________________________________ >> >>> Gluster-users mailing list >> >>> Gluster-users at gluster.org >> >>> http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users >> >>> >> >>> >> >> >> > >> >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users at gluster.org >> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users > > regards, -- Raghavendra G
