Re: rebalance on large filesystems

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Also consider  switching to 'replica 2 arbiter 1' volumes - arbiter can be '192.168.0.3' for the first pair  and '192.168.0.1' for the second.
You will avoid situations with split brain.

Best Regards,
Strahil Nikolov

On Oct 1, 2019 19:15, Strahil <hunter86_bg@xxxxxxxxx> wrote:


On Oct 1, 2019 15:40, ML <lists@websiteburo.com> wrote:
>
> Hi there,
>
> I'm looking for the community's opinion on a gluster volume rebalance
> I'm hestitating to start :)
>
> Here is my config :
>
> Volume Name: nas-3
> Type: Distributed-Replicate
> Status: Started
> Number of Bricks: 2 x 2 = 4
> Transport-type: tcp
> Bricks:
> Brick1: 192.168.0.1:/home/nas-3
> Brick2: 192.168.0.2:/home/nas-3
> Brick3: 192.168.0.3:/home/nas-3
> Brick4: 192.168.0.4:/home/nas-3
> Options Reconfigured:
> performance.quick-read: off
> performance.cache-size: 500MB
> performance.io-thread-count: 32
> transport.address-family: inet
> performance.readdir-ahead: on
>
> The bricks are constitued of 4 gluster servers version 3.10.12 :
>
> 192.168.0.1 + 192.168.0.2 = 2 x 22 TiB (global usage : 86%)
> 192.168.0.3 + 192.168.0.4 = 2 x 7   TiB (global usage : 40%) (this 2
> bricks came 1 year after the 2 first ones, without rebalance after went
> live)
>
> Actual usage :
>
> on 192.168.0.1 + 192.168.0.2 : /home/nas-3 = 2 x 14,5TiB (Items: 49327658)
> on 192.168.0.3 + 192.168.0.4 : /home/nas-3 = 2 x 2,8TiB (Items: 10343981)
>
> I need advice regarding :
>
> - Should I rebalance to fill the 2 x 7.2TiB servers ? (I need space on
> the 22TiB servers)

I would say - yes.

> - Is rebalance process dividing files based on disk usage for each
> connected brick ? (I'm afraid to see my 7.2 TiB servers become full)
Nah... Gluster is way smarter than that.Still, distribution is calculated via an algorithm which we can easily call hashing.Each brick has a hash range and when a file is not on it's hashed location - you should rebalance.

By the way, this should have been done when you added the new bricks.

For details, check: https://staged-gluster-docs.readthedocs.io/en/release3.7.0beta1/Features/dht/

> - Given the number of files stored, how to get an idea of the time
> needed for the rebalance process to be complete ?

You can't estimate that with high accuracy as you got plenty of variables ->  number of files that should be balanced,size of the files that should be balanced, network bandwidth, load on the volume from your clients, speed  of the underlying disks, etc.

It could take a week , maybe less .

> Thank you for reading me & considering answering :))
>
> (If there's a gluster consultant you would recommand for these
> questions, please let me know)
>
> Quentin
>
> ________
>
> Community Meeting Calendar:
>
> APAC Schedule -
> Every 2nd and 4th Tuesday at 11:30 AM IST
> Bridge: https://bluejeans.com/118564314
>
> NA/EMEA Schedule -
> Every 1st and 3rd Tuesday at 01:00 PM EDT
> Bridge: https://bluejeans.com/118564314
>
> Gluster-users mailing list
> Gluster-users@gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-users

Best Regards,
Strahil Nikolov

________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/118564314

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/118564314

Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users

[Index of Archives]     [Gluster Development]     [Linux Filesytems Development]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux