With these 2 changes, we are getting a good improvement in file creation and
slight improvement in the "ls-l" operation.
We are still working to improve the same.
To validate the same we have executed below script from 6 different clients on 24x3 distributed
replicate environment after enabling performance related option
mkdir /gluster-mount/`hostname`
date;
for i in {1..100}
do
echo "directory $i is created" `date`
mkdir /gluster-mount/`hostname`/dir$i
tar -xvf /root/kernel_src/linux-5.4-rc8.tar.xz -C /gluster-mount/`hostname`/dir$i >/dev/null
done
With no Patch
tar was taking almost 36-37 hours
With Patch
tar is taking almost 26 hours
We were getting a similar kind of improvement in smallfile tool also.
To validate the same we have executed below script from 6 different clients on 24x3 distributed
replicate environment after enabling performance related option
mkdir /gluster-mount/`hostname`
date;
for i in {1..100}
do
echo "directory $i is created" `date`
mkdir /gluster-mount/`hostname`/dir$i
tar -xvf /root/kernel_src/linux-5.4-rc8.tar.xz -C /gluster-mount/`hostname`/dir$i >/dev/null
done
With no Patch
tar was taking almost 36-37 hours
With Patch
tar is taking almost 26 hours
We were getting a similar kind of improvement in smallfile tool also.
On Tue, Feb 25, 2020 at 1:29 PM Mohit Agrawal <moagrawa@xxxxxxxxxx> wrote:
Hi,We observed performance is mainly hurt while .glusterfs is having huge data.As we know before executing a fop in POSIX xlator it builds an internal path based on GFID.To validate the path it call's (l)stat system call and while .glusterfs is heavily loaded kernel takes time to lookup inode and due to that performance drops
To improve the same we tried two things with this patch(https://review.gluster.org/#/c/glusterfs/+/23783/)
1) To keep the first level entry always in a cache so that inode lookup will be faster we have to keep open first level fds(00 to ff total 256) per brick at the time of starting a brick process. Even in case of cache cleanup kernel will not evict first level fds from the cache and performance will improve
2) We tried using "at" based call(lstatat,fstatat,readlinat etc) instead of accessing complete path access relative path, these call's were also helpful to improve performance.Regards,Mohit Agrawal
_______________________________________________ Community Meeting Calendar: Schedule - Every Tuesday at 14:30 IST / 09:00 UTC Bridge: https://bluejeans.com/441850968 Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx https://lists.gluster.org/mailman/listinfo/gluster-devel
