Brian,
Small correction: 'sending queries to *both* servers to check they are in sync - even read accesses.' Read fops like stat/getxattr etc are sent to only one brick.
Pranith.
----- Original Message -----
From: "Brian Candler" <B.Candler at pobox.com>
To: "Fernando Frediani (Qube)" <fernando.frediani at qubenet.net>
Cc: "olav johansen" <luxis2012 at gmail.com>, "gluster-users at gluster.org" <gluster-users at gluster.org>
Sent: Thursday, June 7, 2012 4:24:37 PM
Subject: Re: Performance optimization tips Gluster 3.3? (small files / directory listings)
On Thu, Jun 07, 2012 at 10:10:03AM +0000, Fernando Frediani (Qube) wrote:
> Sorry this reply won?t be of any help to your problem, but I am too
> curious to understand how it can be even slower if monting using
> Gluster client which I would expect always be quicker than NFS or
> anything else.
(1) Try it with "ls -aR" or "find ." instead of "ls -alR"
(2) Try it on a gluster non-replicated volume (for fair comparison with
direct NFS access)
With a replicated volume, many accesses involve sending queries to *both*
servers to check they are in sync - even read accesses. This in turn can
cause disk seeks on both machines, so the latency you'll get is the larger
of the two. If you are doing lots of accesses sequentially then the
latencies will all add up.
A stat() is one of those accesses which touches both machines, and "ls -l"
forces a stat() of each file found.
In fact, a quick test suggests ls -l does stat, lstat, getxattr and
lgetxattr:
$ ls -laR . >/dev/null 2>ert; cut -f1 -d'(' ert | sort | uniq -c
13 access
1 arch_prctl
5 brk
395 close
4 connect
1 execve
1 exit_group
2 fcntl
391 fstat
3 futex
702 getdents
1 getrlimit
1719 getxattr
3 ioctl
1721 lgetxattr
9 lseek
1721 lstat
58 mmap
24 mprotect
12 munmap
424 open
19 read
2 readlink
2 rt_sigaction
1 rt_sigprocmask
1 set_robust_list
1 set_tid_address
4 socket
1719 stat
1 statfs
29 write
Looking at the detail in the strace output, I see these are actually
lstat(<target-file>, ...)
lgetxattr(<target-file>, "security.selinux", ...)
getxattr(<target-file>, "system.posix_acl_access", ...)
stat("/etc/localtime", ...)
Compare without -l:
$ strace ls -aR . >/dev/null 2>ert; cut -f1 -d'(' ert | sort | uniq -c
9 access
1 arch_prctl
4 brk
377 close
1 execve
1 exit_group
1 fcntl
376 fstat
3 futex
702 getdents
1 getrlimit
3 ioctl
39 mmap
16 mprotect
4 munmap
388 open
11 read
2 rt_sigaction
1 rt_sigprocmask
1 set_robust_list
1 set_tid_address
1 stat
1 statfs
9 write
Regards,
Brian.
_______________________________________________
Gluster-users mailing list
Gluster-users at gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
