Hi I'm having a weird issue where my gluster volumes are being reported as full by df but du shows only 18GB of actual data in them (which is less than 1% of the capacity of the volumes). The client shows 2/3 of the volume is full, only replicate-3 is currently not "full" and given how this problem started I expect it to be full by the time I come into the office tomorrow morning. I am runnung glusterfs 3.1.2 on Ubuntu 10.10. Does anyone have any idea why this might be happening? # df -h Filesystem Size Used Avail Use% Mounted on /dev/md3 1.7T 1.7T 0 100% /export/glusterfs Output of dh in the exported folder on my first gluster host # du -h .. ... .... 18G total # cat /etc/glusterd/vols/backup-volume/backup-volume-fuse.vol volume backup-volume-client-0 type protocol/client option remote-host gluster01 option remote-subvolume /export/glusterfs option transport-type tcp end-volume volume backup-volume-client-1 type protocol/client option remote-host gluster02 option remote-subvolume /export/glusterfs option transport-type tcp end-volume volume backup-volume-client-2 type protocol/client option remote-host gluster03 option remote-subvolume /export/glusterfs option transport-type tcp end-volume volume backup-volume-client-3 type protocol/client option remote-host gluster04 option remote-subvolume /export/glusterfs option transport-type tcp end-volume volume backup-volume-client-4 type protocol/client option remote-host gluster05 option remote-subvolume /export/glusterfs option transport-type tcp end-volume volume backup-volume-client-5 type protocol/client option remote-host gluster06 option remote-subvolume /export/glusterfs option transport-type tcp end-volume volume backup-volume-client-6 type protocol/client option remote-host gluster07 option remote-subvolume /export/glusterfs option transport-type tcp end-volume volume backup-volume-client-7 type protocol/client option remote-host gluster08 option remote-subvolume /export/glusterfs option transport-type tcp end-volume volume backup-volume-client-8 type protocol/client option remote-host gluster09 option remote-subvolume /export/glusterfs option transport-type tcp end-volume volume backup-volume-replicate-0 type cluster/replicate subvolumes backup-volume-client-0 backup-volume-client-1 backup-volume-client-2 end-volume volume backup-volume-replicate-1 type cluster/replicate subvolumes backup-volume-client-3 backup-volume-client-4 backup-volume-client-5 end-volume volume backup-volume-replicate-2 type cluster/replicate subvolumes backup-volume-client-6 backup-volume-client-7 backup-volume-client-8 end-volume volume backup-volume-dht type cluster/distribute subvolumes backup-volume-replicate-0 backup-volume-replicate-1 backup-volume-replicate-2 end-volume volume backup-volume-write-behind type performance/write-behind subvolumes backup-volume-dht end-volume volume backup-volume-read-ahead type performance/read-ahead subvolumes backup-volume-write-behind end-volume volume backup-volume-io-cache type performance/io-cache subvolumes backup-volume-read-ahead end-volume volume backup-volume-quick-read type performance/quick-read subvolumes backup-volume-io-cache end-volume volume backup-volume-stat-prefetch type performance/stat-prefetch subvolumes backup-volume-quick-read end-volume volume backup-volume type debug/io-stats subvolumes backup-volume-stat-prefetch end-volume