Re: Rebalance improvement.

Susant Palai <spalai@xxxxxxxxxx> · Mon, 3 Aug 2020 14:09:37 +0530

On 03-Aug-2020, at 13:58, Aravinda VK <aravinda@xxxxxxxxx> wrote:

Interesting numbers. Thanks for the effort.
What is the unit of old/new numbers? seconds? 

Minutes. 

On 03-Aug-2020, at 12:47 PM, Susant Palai <spalai@xxxxxxxxxx> wrote:

Centos Users can add the following repo and install the build from the master branch to try out the feature. [Testing purpose only, not ready for consumption in production env.]
[gluster-nightly-master]
baseurl=http://artifacts.ci.centos.org/gluster/nightly/master/7/x86_64/
gpgcheck=0
keepalive=1
enabled=1
repo_gpgcheck = 0
name=Gluster Nightly builds (master branch)

A summary of perf numbers from our test lab :

DirSize - 1Million	Old	New	%diff
Depth 	- 100 (Run 1)	353	74	+377%
Depth 	- 100 (Run 2)	348	72	+377~%
Depth 	- 50		246	122	+100%
Depth 	- 3		174	114	+52%

Susant

On Mon, Aug 3, 2020 at 11:16 AM Susant Palai <spalai@xxxxxxxxxx> wrote:
Hi,    Recently, we have pushed some performance improvements for Rebalance Crawl which used to consume a significant amount of time, out of the entire rebalance process.

The patch [1] is recently merged in upstream and may land as an experimental feature in the upcoming upstream release.

The improvement currently works only for pure-distribute Volume. (which can be expanded).

Things to look forward to in future :
 - Parallel Crawl in Rebalance
 - Global Layout

Once these improvements are in place, we would be able to reduce the overall rebalance time by a significant time.

Would request our community to try out the feature and give us feedback.

More information regarding the same will follow.

Thanks & Regards,
Susant Palai

[1] https://review.gluster.org/#/c/glusterfs/+/24443/
________

Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://bluejeans.com/441850968

Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users

Aravinda Vishwanathapura
https://kadalu.io

________

Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://bluejeans.com/441850968

Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users