Re: df shows wrong mount size, after adding bricks to volume

[Sanju Rakonde <srakonde@xxxxxxxxxx>]

The shared-brick-count value indicates the number of bricks sharing a file-system. In your case, it should be one, as all the bricks are from different mount points. Can you please share the values of brick-fsid?

grep "brick-fsid" /var/lib/glusterd/vols/<volname>/bricks/

I tried reproducing this issue in fedora vm's but couldn't hit this. we are seeing this issue on and off but are unable to reproduce in-house. If you see any error messages in glusterd.log please share the log too.

Work-around to come out from this situation:

1. Restarting the glusterd service on all nodes:
# systemctl restart glusterd

2. Run set volume command to update vol file:
# gluster v set <VOLNAME> min-free-disk 11%

On Wed, May 27, 2020 at 5:24 PM Petr Certik <petr@xxxxxxxxx> wrote:

As far as I remember, there was no version update on the server. It
was definitely installed as version 7.

Shared bricks:

Server 1:

/var/lib/glusterd/vols/gv0/gv0.imagegluster1.data2-brick.vol:
option shared-brick-count 2
/var/lib/glusterd/vols/gv0/gv0.imagegluster1.data-brick.vol:    option
shared-brick-count 2
/var/lib/glusterd/vols/gv0/gv0.imagegluster2.data2-brick.vol:
option shared-brick-count 0
/var/lib/glusterd/vols/gv0/gv0.imagegluster2.data-brick.vol:    option
shared-brick-count 0
/var/lib/glusterd/vols/gv0/gv0.imagegluster3.data2-brick.vol:
option shared-brick-count 0
/var/lib/glusterd/vols/gv0/gv0.imagegluster3.data-brick.vol:    option
shared-brick-count 0

Server 2:

/var/lib/glusterd/vols/gv0/gv0.imagegluster1.data2-brick.vol:
option shared-brick-count 0
/var/lib/glusterd/vols/gv0/gv0.imagegluster1.data-brick.vol:    option
shared-brick-count 0
/var/lib/glusterd/vols/gv0/gv0.imagegluster2.data2-brick.vol:
option shared-brick-count 2
/var/lib/glusterd/vols/gv0/gv0.imagegluster2.data-brick.vol:    option
shared-brick-count 2
/var/lib/glusterd/vols/gv0/gv0.imagegluster3.data2-brick.vol:
option shared-brick-count 0
/var/lib/glusterd/vols/gv0/gv0.imagegluster3.data-brick.vol:    option
shared-brick-count 0

Server 3:

/var/lib/glusterd/vols/gv0/gv0.imagegluster1.data2-brick.vol:
option shared-brick-count 0
/var/lib/glusterd/vols/gv0/gv0.imagegluster1.data-brick.vol:    option
shared-brick-count 0
/var/lib/glusterd/vols/gv0/gv0.imagegluster2.data2-brick.vol:
option shared-brick-count 0
/var/lib/glusterd/vols/gv0/gv0.imagegluster2.data-brick.vol:    option
shared-brick-count 0
/var/lib/glusterd/vols/gv0/gv0.imagegluster3.data2-brick.vol:
option shared-brick-count 2
/var/lib/glusterd/vols/gv0/gv0.imagegluster3.data-brick.vol:    option
shared-brick-count 2

On Wed, May 27, 2020 at 1:36 PM Sanju Rakonde <srakonde@xxxxxxxxxx> wrote:
>
> Hi Petr,
>
> what was the server version before upgrading to 7.2?
>
> Can you please share the shared-brick-count values from brick volfiles from all the nodes?
> grep shared-brick-count /var/lib/glusterd/vols/<volume_name>/*
>
> On Wed, May 27, 2020 at 2:31 PM Petr Certik <petr@xxxxxxxxx> wrote:
>>
>> Hi everyone,
>>
>> we've been running a replicated volume for a while, with three ~1 TB
>> bricks. Recently we've added three more same-sized bricks, making it a
>> 2 x 3 distributed replicated volume. However, even after rebalance,
>> the `df` command on a client shows the correct used/size percentage,
>> but wrong absolute sizes. The size still shows up as ~1 TB while in
>> reality it should be around 2 TB, and both "used" and "available"
>> reported sizes are about half of what they should be. The clients were
>> an old version (5.5), but even after upgrade to 7.2 and remount, the
>> reported sizes are still wrong. There are no heal entries. What can I
>> do to fix this?
>>
>> OS: debian buster everywhere
>> Server version: 7.3-1, opversion: 70200
>> Client versions: 5.5-3, 7.6-1, opversions: 50400, 70200
>>
>>
>> root@imagegluster1:~# gluster volume info gv0
>> Volume Name: gv0
>> Type: Distributed-Replicate
>> Volume ID: 5505d350-9b61-4056-9054-de9dfb58eab7
>> Status: Started
>> Snapshot Count: 0
>> Number of Bricks: 2 x 3 = 6
>> Transport-type: tcp
>> Bricks:
>> Brick1: imagegluster1:/data/brick
>> Brick2: imagegluster2:/data/brick
>> Brick3: imagegluster3:/data/brick
>> Brick4: imagegluster1:/data2/brick
>> Brick5: imagegluster2:/data2/brick
>> Brick6: imagegluster3:/data2/brick
>> Options Reconfigured:
>> features.cache-invalidation: on
>> transport.address-family: inet
>> storage.fips-mode-rchecksum: on
>> nfs.disable: on
>> performance.client-io-threads: off
>>
>>
>> root@imagegluster1:~# df -h
>> Filesystem      Size  Used Avail Use% Mounted on
>> ...
>> /dev/sdb1       894G  470G  425G  53% /data2
>> /dev/sdc1       894G  469G  426G  53% /data
>>
>>
>> root@any-of-the-clients:~# df -h
>> Filesystem         Size  Used Avail Use% Mounted on
>> ...
>> imagegluster:/gv0  894G  478G  416G  54% /mnt/gluster
>>
>>
>> Let me know if there's any other info I can provide about our setup.
>>
>> Cheers,
>> Petr Certik
>> ________
>>
>>
>>
>> Community Meeting Calendar:
>>
>> Schedule -
>> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
>> Bridge: https://bluejeans.com/441850968
>>
>> Gluster-users mailing list
>> Gluster-users@xxxxxxxxxxx
>> https://lists.gluster.org/mailman/listinfo/gluster-users
>>
>
>
> --
> Thanks,
> Sanju

--

Thanks,

Sanju

________



Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://bluejeans.com/441850968

Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users