Hey Everyone,
I have a 3 node GlusterFS cluster that uses NVMe hot tier and a HDD cold tier.
I recently ran into some problems when the hot tier became full with df-h showing 100%.
I did not have a watermark-hi set, but it is my understanding that 90% is the default. In an attempt to get the cluster to demote some files, I set cluster.watermark-hi: 80 but it is still not demoting.
[root@Glus1 ~]# gluster volume info
Volume Name: FFPrimary
Type: Tier
Volume ID: 466ec53c-d1ef-4ebc-8414-d7d070dfe61e
Status: Started
Snapshot Count: 0
Number of Bricks: 9
Transport-type: tcp
Hot Tier :
Hot Tier Type : Replicate
Number of Bricks: 1 x 3 = 3
Brick1: Glus3:/data/glusterfs/FFPrimary/brick3
Brick2: Glus2:/data/glusterfs/FFPrimary/brick2
Brick3: Glus1:/data/glusterfs/FFPrimary/brick1
Cold Tier:
Cold Tier Type : Distributed-Replicate
Number of Bricks: 2 x 3 = 6
Brick4: Glus1:/data/glusterfs/FFPrimary/brick5
Brick5: Glus2:/data/glusterfs/FFPrimary/brick6
Brick6: Glus3:/data/glusterfs/FFPrimary/brick7
Brick7: Glus1:/data/glusterfs/FFPrimary/brick8
Brick8: Glus2:/data/glusterfs/FFPrimary/brick9
Brick9: Glus3:/data/glusterfs/FFPrimary/brick10
Options Reconfigured:
cluster.tier-promote-frequency: 120
cluster.tier-demote-frequency: 1800
cluster.watermark-low: 60
cluster.watermark-hi: 80
performance.flush-behind: on
performance.cache-max-file-size: 128MB
performance.cache-size: 25GB
diagnostics.count-fop-hits: off
diagnostics.latency-measurement: off
cluster.tier-mode: cache
features.ctr-enabled: on
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
[root@Glus1 ~]# gluster volume tier FFPrimary status
Node Promoted files Demoted files Status run time in h:m:s
--------- --------- --------- --------- ---------
localhost 49 0 in progress 5151:30:45
Glus2 0 0 in progress 5151:30:45
Glus3 0 2075 in progress 5151:30:47
Tiering Migration Functionality: FFPrimary: success
[root@Glus1 ~]#
What can cause GlusterFS to stop demoting files and allow it to completely fill the Hot Tier?
Thank you!
_______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx https://lists.gluster.org/mailman/listinfo/gluster-users
