Wrong directory quota usage

[João Baúto <joao.bauto@xxxxxxxxxxxxxxxxxxxxxxx>]

Hi all,

We have a 4-node distributed cluster with 2 bricks per node running Gluster 7.7 + ZFS. We use directory quota to limit the space used by our members on each project. Two days ago we noticed inconsistent space used reported by Gluster in the quota list.

A small snippet of gluster volume quota vol list,

 Path                   Hard-limit  Soft-limit          Used            Available         Soft-limit exceeded?   Hard-limit exceeded?

/projectA              5.0TB        80%(4.0TB)    3.1TB           1.9TB              No                               No
/projectB            100.0TB    80%(80.0TB)  16383.4PB   740.9TB         No                               No
/projectC              70.0TB     80%(56.0TB)   50.0TB         20.0TB           No                              No

The total space available in the cluster is 360TB, the quota for projectB is 100TB and, as you can see, its reporting 16383.4PB used and 740TB available (already decreased from 750TB).

There was an issue in Gluster 3.x related to the wrong directory quota (https://lists.gluster.org/pipermail/gluster-users/2016-February/025305.html and https://lists.gluster.org/pipermail/gluster-users/2018-November/035374.html) but it's marked as solved (not sure if the solution still applies).

On projectB

# getfattr -d -m . -e hex projectB
# file: projectB
trusted.gfid=0x3ca2bce0455945efa6662813ce20fc0c
trusted.glusterfs.9582685f-07fa-41fd-b9fc-ebab3a6989cf.xtime=0x5f35e69800098ed9
trusted.glusterfs.dht=0xe1a4060c000000003ffffffe5ffffffc
trusted.glusterfs.mdata=0x010000000000000000000000005f355c59000000000939079f000000005ce2aff90000000007fdacb0000000005ce2aff90000000007fdacb0
trusted.glusterfs.quota.00000000-0000-0000-0000-000000000001.contri.1=0x0000ab0f227a860000000000478e33acffffffffffffc112
trusted.glusterfs.quota.dirty=0x3000
trusted.glusterfs.quota.limit-set.1=0x0000640000000000ffffffffffffffff
trusted.glusterfs.quota.size.1=0x0000ab0f227a860000000000478e33acffffffffffffc112

On projectA

# getfattr -d -m . -e hex projectA
# file: projectA
trusted.gfid=0x05b09ded19354c0eb544d22d4659582e
trusted.glusterfs.9582685f-07fa-41fd-b9fc-ebab3a6989cf.xtime=0x5f1aeb9f00044c64
trusted.glusterfs.dht=0xe1a4060c000000001fffffff3ffffffd
trusted.glusterfs.mdata=0x010000000000000000000000005f1ac6a10000000018f30a4e000000005c338fab0000000017a3135a000000005b0694fb000000001584a21b
trusted.glusterfs.quota.00000000-0000-0000-0000-000000000001.contri.1=0x0000067de3bbe20000000000000128610000000000033498
trusted.glusterfs.quota.dirty=0x3000
trusted.glusterfs.quota.limit-set.1=0x0000460000000000ffffffffffffffff
trusted.glusterfs.quota.size.1=0x0000067de3bbe20000000000000128610000000000033498

Any idea on what's happening and how to fix it?

Thanks!

João Baúto

---------------

Scientific Computing and Software Platform

Champalimaud Research
Champalimaud Center for the Unknown
Av. Brasília, Doca de Pedrouços
1400-038 Lisbon, Portugal
fchampalimaud.org

________



Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://bluejeans.com/441850968

Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users