Re: Readdirp (ls -l) Performance Improvement

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Sure, I have back-ported the patch to release-7. Now I will see How I can build the rpms.

On the other hand, if possible, Could you please tell me about the volume configuration and the volume tree structure. This will be more helpful when you have lot of files or a distribution is high.

Regards

Rafi KC

On 27/05/20 5:49 pm, Strahil Nikolov wrote:
Hi Rafi,

I have a test oVirt 4.3.9 cluster  with Gluster v7.5 on CentOS7.
Can you provide the rpms and I will try to test.

Also, please share the switch that disables this behaviour (in case something goes wrong).

Best Regards,
Strahil Nikolov

На 27 май 2020 г. 14:54:34 GMT+03:00, RAFI KC <rkavunga@xxxxxxxxxx> написа:
Hi Felix,

If you already have a test cluster, please give me the gluster version.

I will try to build rpm then you can just do an upgrade. If you don't
have a test cluster, and you are creating a new one, then I will build
a
new rpm against the latest head. This is more easier to me as I don't
have to backport to a previous version.

May I ask you the access protocol that you use (FUSE, SMB , NFS,
gfapi).
I did the testing on a fuse protocol.


Rafi KC

On 27/05/20 4:14 pm, Felix Kölzow wrote:
Dear Rafi KC,


lets suppose I going to spend some time for testing. How would I
install glusterfs-server including your feature?

Maybe this is an easy procedure, but actually I am not familiar with
it.
Regards,

Felix

On 27/05/2020 07:56, RAFI KC wrote:
Hi All,

I have been working on POC to improve readdirp performance
improvement. At the end of the experiment, The results are showing
promising result in performance, overall there is a 104% improvement
for full filesystem crawl compared to the existing solution. Here is
the short test numbers. The tests were carried out in 16*3 setup
with
1.5 Million dentries (Both files and dir). The system also contains
some empty directories. *In the result the proposed solution is 287%
faster than the plane volume and 104% faster than the
parallel-readdir based solution.*

*
*

Configuration

	

Plain volume

	

Parallel-readdir

	

Proposed Solution

FS Crawl Time in Seconds

	

16497.523

	

8717.872

	

4261.401

In short, the basic idea behind the proposal is the efficient
managing of readdir buffer in gluster along with prefetching the
dentries for intelligent switch-over to the next buffer. The
detailed
problem description, deign description and results are available in
the

doc.https://docs.google.com/document/d/10z4T5Sd_-wCFrmDrzyQtlWOGLang1_g17wO8VUxSiJ8/edit



If anybody can help with the testing on a different kind of
workloads, I would be very happy to assist. If wanted to test the
patch and run a performance test on your setup, I could help with
back-porting the patch to the version of your choice.


https://review.gluster.org/24469

https://review.gluster.org/24470


Regards

Rafi KC


________



Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge:https://bluejeans.com/441850968

Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users
________



Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://bluejeans.com/441850968

Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users

________



Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://bluejeans.com/441850968

Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users




[Index of Archives]     [Gluster Development]     [Linux Filesytems Development]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux