On Mon, 3 Mar 2008, Anand Avati wrote:
the bottleneck seems to be create/open which is synchronous over all subvolumes. do you have numbers without involving the remote site/afr ?
Just mounted nyc rather then mirror and that cut the time to .076 sec vs local .01 sec.
This is a 100 meg link between NYC and SJC and only transferring 748K. My guess is even if we had a full gig link between the two we would have the same problem. A message sent from NYC to SJC will still take 60ms to fly across the country. So if you have a lot of messages flying back and forth waiting for data to be written you will have this problem.
Another clue is that it takes 27 sec just to rm -fr the files! If I moved afr from the client to the server would that help?
[root@xen0 glusterfs]# time cp -r /etc/sysconfig/ /share/mirror real 0m0.076s user 0m0.000s sys 0m0.007s [root@xen0 glusterfs]# time cp -r /etc/sysconfig/ /share/ real 0m0.010s user 0m0.001s sys 0m0.006s [root@xen0 glusterfs]# With mirror mounted (sjc and nyc up): [root@xen0 glusterfs]# time cp -r /etc/sysconfig/ /share/mirror real 0m52.195s user 0m0.000s sys 0m0.001s [root@xen0 glusterfs]# time rm -fr /share/mirror/* real 0m27.030s user 0m0.000s sys 0m0.002s
<>
Nathan Stratton nathan at robotics.net http://www.robotics.net
