Hi John I got this yesterday, it was copied to this list. Cheers, On Tue, 2014-06-17 at 04:55 -0400, Susant Palai wrote: Hi Franco: > The following patches address the ENOTEMPTY issue. > > 1. http://review.gluster.org/#/c/7733/ > 2. http://review.gluster.org/#/c/7599/ > > I think the above patches will be available in 3.5.1 which will be a minor upgrade.(Need ack from Niels de Vos.) > On Wed, 2014-06-18 at 13:26 +0930, SINCOCK John wrote: > Thanks Pranith, I hope Susant can help! > > > > From: Pranith Kumar Karampuri [mailto:pkarampu@xxxxxxxxxx] > Sent: Wednesday, 18 June 2014 11:48 AM > To: SINCOCK John; gluster-users@xxxxxxxxxxx > Cc: Susant Palai > Subject: Re: Unable to delete files but getfattr shows > file is part of glusterfs > > > > > I think Susant knows about this issue > > CCed him > > Pranith > > On 06/15/2014 07:16 PM, SINCOCK John wrote: > > > Hi Everybody, > > > > We have recently setup a 2-node gluster, and are having some > problems deleting some files! > > We haven’t accessed the gluster bricks directly, all access > has been through the glusterfs mount > > The 1st node in our cluster also handles sharing the glusterfs > volume via samba, to windows clients. > > > > Environment is: > > Kernel: 2.6.32-431.1.2.0.1.el6.x86_64 #1 SMP Fri Dec 13 > 13:06:13 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux > > CentOS release 6.4 (Final) > > > > glusterfs-libs-3.4.1-3.el6.x86_64 > > glusterfs-cli-3.4.1-3.el6.x86_64 > > glusterfs-fuse-3.4.1-3.el6.x86_64 > > glusterfs-server-3.4.1-3.el6.x86_64 > > glusterfs-3.4.1-3.el6.x86_64 > > > > 4 bricks on each node > > Gluster is setup without replication or > striping > > > > Bricks on one of the two nodes (the other node > is setup the same): > > /dev/sda1 9.1T 4.7T 4.5T > 51% /mnt/glusterfs/bricks/1 > > /dev/sda2 9.1T 3.7T 5.5T > 40% /mnt/glusterfs/bricks/2 > > /dev/sda3 9.1T 3.7T 5.5T > 40% /mnt/glusterfs/bricks/3 > > /dev/sda4 9.1T 4.1T 5.0T > 46% /mnt/glusterfs/bricks/4 > > > > Gluster volume mounted on 1st node: > > g-unit-1:/vol00 73T 31T 43T 42% /gluster/vol00 > > Mount options: > > g-unit-1:/vol00 on /gluster/vol00 type fuse.glusterfs > (rw,nosuid,nodev,noatime,default_permissions,allow_other,max_read=131072) > > > > So the problem is this: > > [root@g-unit-1 ~]# rm > -Rf /gluster/vol00/data/CentOS3/CentOS_6.4_Final > > rm: cannot remove > `/gluster/vol00/data/CentOS3/CentOS_6.4_Final/Packages': > Directory not empty > > rm: cannot remove > `/gluster/vol00/data/CentOS3/CentOS_6.4_Final/EFI/BOOT': > Directory not empty > > [root@g-unit-1 ~]# > > > > And the delete leaves behind 2000 or so files: > > [root@g-unit-1 ~]# > find /gluster/vol00/data/CentOS3/CentOS_6.4_Final/ -type f | > wc -l > > 2185 > > Almost all the left behind files are on the 2nd node, > g-unit-2, but there are a few on g-unit-1 > > > > From the 2nd node, g-unit-2: > > [root@g-unit-2 ~]# getfattr -m . -d -e > hex /mnt/glusterfs/bricks/2/data/CentOS3/CentOS_6.4_Final/EFI/BOOT/BOOTX64.efi > > getfattr: Removing leading '/' from absolute path names > > # file: > mnt/glusterfs/bricks/2/data/CentOS3/CentOS_6.4_Final/EFI/BOOT/BOOTX64.efi > > trusted.gfid=0xa9b0a66cf39b49c7890b2da3f631929f > > So the file is there, is tagged with some gluster info, and > has definitely been copied onto the volume properly, via the > glusterfs mount. It hasn’t been copied directly onto the > brick. > > > > But from g-unit-1, accessing via the glusterfs mount, and > trying to delete this file: > > [root@g-unit-1 ~]# > rm /gluster/vol00/data/CentOS3/CentOS_6.4_Final/EFI/BOOT/BOOTX64.efi > > rm: remove regular file > `/gluster/vol00/data/CentOS3/CentOS_6.4_Final/EFI/BOOT/BOOTX64.efi'? y > > rm: cannot remove > `/gluster/vol00/data/CentOS3/CentOS_6.4_Final/EFI/BOOT/BOOTX64.efi': No such file or directory > > > > What the hell!?!!!!! > > > > Can someone please suggest the best course of action to > resolve this and prevent it happening in the future? > > > > Thanks for any assistance. > > > > John > > > > > > > > > > > > > > > > > _______________________________________________ > Gluster-users mailing list > Gluster-users@xxxxxxxxxxx > http://supercolony.gluster.org/mailman/listinfo/gluster-users > > > > > _______________________________________________ > Gluster-users mailing list > Gluster-users@xxxxxxxxxxx > http://supercolony.gluster.org/mailman/listinfo/gluster-users _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://supercolony.gluster.org/mailman/listinfo/gluster-users
