Hello, We are trying to configure gluster replication for a cyrus imap email server. Even after heavy tweaking configs for small files, (bug 314 was solved at 2.0.8 but i did not backported it to debian lenny yet), the performance for parallel executions over the fs are VERY painfully slow, for a 3 mirrors set. We used, instead of the cyrus server, for preliminary simple tests, the postmark sw and some shell scripting, before advancing for bonnie++. The cyrus behaviour fs use is "conceptually similar". The parallel shell execution: http://techie-mee.blogspot.com/2008/05/run-shell-commands-in-parallel.html It is a slow shell script. But enough for some preliminary comparison tests. Please, read below. I removed the long verbose parallel log output. Take a look at times and quantities. The script for removing long list of files is simple. Only a loop of rm by line output of ls. Read the time difference for removing files sequentially or in parallel calls. Also, for comparison, see at an ext3 local directory. The parallel shell script is slow, but performs much faster at local ext3 directory. This very poor speed behaviour is seen as the server spawns many requests to the filesystem also. What could be done? Any suggestions? Regards. Andre Felipe Machado http://www.techforce.com.br debian459140:~# mount -t glusterfs -o suid,noatime /root/arquivos_glusterfs/testes/glusterfs.vol.afr.tcpnodelay.noreadahead.statprefetch.laboratorio /mnt/glusterfs debian459140:/mnt/glusterfs# yes | head -c 100K | split -b 1 -a 6 debian459140:~# ls /mnt/glusterfs | wc -l 102400 debian459140:~# find /mnt/glusterfs -type f | sed -e 's/\(.*\)/rm \1/g' > tmp.txt debian459140:~# time cat tmp.txt | sh parallel.sh 8 aborted due slow speed real 14m21.131s user 0m0.784s sys 0m1.008s debian459140:/mnt/glusterfs# ls /mnt/glusterfs |wc -l 102239 debian459140:/mnt/glusterfs# debian459140:/mnt/glusterfs# time sh /root/arquivos_glusterfs/testes/remove_millions_files_mnt_glusterfs.sh real 7m37.446s user 0m18.681s sys 1m43.238s debian459140:/mnt/glusterfs# yes | head -c 100K | split -b 1 -a 6 debian459140:~# find /root/temp -type f | sed -e 's/\(.*\)/rm \1/g' > tmp.txt debian459140:~# tail tmp.txt rm /root/temp/xaafoel rm /root/temp/xaafnuv rm /root/temp/xaabjzy rm /root/temp/xaaeggu rm /root/temp/xaaeyqk rm /root/temp/xaabque rm /root/temp/xaabkhg rm /root/temp/xaafrsp rm /root/temp/xaaerzj rm /root/temp/xaadgom debian459140:~# time cat tmp.txt | sh parallel.sh 8 real 62m26.245s user 73m17.499s sys 45m54.600s debian459140:~#
