Anand Avati wrote:
You might also be interested in mod_glusterfs. This is an apache module
which lets you run the filesystem inside apache address space (in a seperate
thread) which can greatly boost the performance of serving static files.
This module is available if you checkout glusterfs--mainline--3.0. Currently
mod_glusterfs works with apache-1.3.x, but work for getting it ready for
apache2.x and lighttpd is on the way. The way to use it - (in your
httpd.conf)
That sounds great, however I'm currently bound to Lighttpd.
I've setup a clean test case to benchmark the issues I'm seeing. With -e
600 -a 600 it seems I'm no longer bound to stat(), but to open() now.
Following is a trace of a "Hello World" PHP script (FCGI) as well as my
current configuration.
I think I messed something with iothreads. Without GlusterFS total
throughput increases a lot with multiple HTTP clients, with GlusterFS it
decreases. I could not find any load where using iothreads improved my
situation, so there still seems to be a single point of serialisation.
If you have any suggestion on how to improve this configuration, I'd
really appreciative it.
Thanks in advance.
Strace with GlusterFS:
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
43.99 0.053319 5 10000 open
16.22 0.019654 2 10000 accept
12.95 0.015696 1 20000 close
7.66 0.009287 0 60000 read
6.73 0.008154 0 20000 fstat
3.53 0.004273 0 10000 write
1.71 0.002071 0 10000 munmap
1.61 0.001955 0 50000 gettimeofday
1.33 0.001614 0 40000 fcntl
1.07 0.001301 0 10000 mmap
1.01 0.001222 0 10000 chdir
0.51 0.000620 0 30000 setitimer
0.45 0.000551 0 20000 recvfrom
0.36 0.000442 0 10000 shutdown
0.26 0.000313 0 20000 rt_sigaction
0.25 0.000302 0 10000 select
0.22 0.000261 0 20000 rt_sigprocmask
0.14 0.000164 0 10000 lseek
------ ----------- ----------- --------- --------- ----------------
100.00 0.121199 370000 total
Strace without GlusterFS (XFS noatime, lazy-counters):
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
34.98 0.008714 1 10000 accept
13.22 0.003293 0 60000 read
9.41 0.002345 0 10000 write
7.59 0.001890 0 10000 open
6.81 0.001696 0 50000 gettimeofday
5.34 0.001331 0 40000 fcntl
5.07 0.001264 0 20000 close
3.11 0.000774 0 10000 munmap
2.77 0.000690 0 10000 chdir
1.92 0.000478 0 20000 recvfrom
1.91 0.000476 0 20000 fstat
1.79 0.000446 0 30000 setitimer
1.60 0.000399 0 10000 mmap
1.09 0.000271 0 20000 rt_sigaction
1.04 0.000260 0 10000 lseek
0.93 0.000231 0 20000 rt_sigprocmask
0.77 0.000191 0 10000 select
0.65 0.000161 0 10000 shutdown
------ ----------- ----------- --------- --------- ----------------
100.00 0.024910 370000 total
The server config is:
volume webroot
type storage/posix
option directory /data/webs/default/webroot_server
end-volume
volume src
type performance/io-threads
option thread-count 8
option cache-size 32MB
subvolumes webroot
end-volume
volume server
type protocol/server
option transport-type tcp/server
subvolumes webroot
option auth.ip.webroot.allow *
end-volume
The client config:
volume client
type protocol/client
option transport-type tcp/client
option remote-host localhost
option transport-timeout 10
option remote-subvolume webroot
end-volume
volume iot
type performance/io-threads
option thread-count 8
option cache-size 32MB
subvolumes client
end-volume
volume wb
type performance/write-behind
option aggregate-size 1MB
option flush-behind on
subvolumes iot
end-volume
volume io-cache
type performance/io-cache
option cache-size 64MB
option page-size 128KB
option force-revalidate-timeout 600
subvolumes wb
end-volume
--
Best regards,
Hannes Dorbath
