I too am searching for a way to get Gluster to use a cache effectively.
So far, no performance translator, variation on translator parameters,
or locating translator on server or clients side has made any
difference. :(
Jeremy
On 3/31/2010 3:23 AM, Olivier Le Cam wrote:
> Hi -
>
>> * Can you test dd performance with only write-behind translator
>> (along with
>> protocol/client) in client volume configuration?
>
> That makes no difference with either only write-behind translator or
> all of default translator configurated by genvol. Without it, results
> are abt 10 times worse.
>
>> * Also can you add "debug/trace" translator above and below
>> write-behind,
>> start glusterfs with Log-level trace and send the log files?
>
> * Debian/Lenny (kernel 2.6.32-trunk-amd64, same with 2.6.26)
> ~# dd if=/dev/zero of=/mnt/test_file count=262144 bs=1024
> 262144+0 enregistrements lus
> 262144+0 enregistrements ??crits
> 268435456 bytes (268 MB) copied, 117,14 s, 2,3 MB/s
>
> Log file size is 283MB. Do you want to get it entirely?
> First 10000 lines of /var/log/glusterfs/mnt.log uploaded to:
> http://olecam.online.fr/glusterfs/lenny_log.txt
>
> ~# cat /var/log/glusterfs/mnt.log |wc -l
> 1792172
>
>
> * Ubuntu/Hardy (kernel 2.6.24-23-server x86_64)
> ~# dd if=/dev/zero of=/mnt/test_file count=262144 bs=1024
> 262144+0 records in
> 262144+0 records out
> 268435456 bytes (268 MB) copied, 17,1914 s, 15,6 MB/s
>
> Log file size is 115MB. Do you want to get it entirely?
> First 10000 lines of /var/log/glusterfs/mnt.log uploaded to:
> http://olecam.online.fr/glusterfs/ubuntu_log.txt
>
> ~# cat /var/log/glusterfs/mnt.log |wc -l
> 553600
>
> * Addition notes:
>
> - Using tcpdump, I have observed that dd'ing from the Debian/Lenny
> generates abt 10 times more frames on the network. Frame size sent by
> the Ubuntu client are generally larger, this can explain why the
> transfer rate is so terrible from my Lenny client.
>
> - tcpdump on Lenny client shows: "278648 packets captured"
> - tcpdump on Ubuntu client shows: "28120 packets captured" (abt 10x
> less frames traffic!!)
>
> tcpdump -v outputs also show that packet size sent the Ubuntu client
> is usually abt 40-60 KBytes large. For example:
> 10:06:34.795101 IP (tos 0x0, ttl 64, id 38433, offset 0, flags [DF],
> proto TCP (6), length 50732)
> This is never the case from the Debian client(always < 10KBytes).
>
> As observed by Paul who has better results than mine with an Lenny
> client, this might be related to some kernel options or
> "optimizations" in recent versions. I haven't been able to give a test
> with 2.6.18, the only one I have under hand is a xen domU (under an
> Etch dom0) and I cannot get the fuse loading properly on it.
>
> Still available for anything else I can do.
>
> Thanks and best regards,
