So I'm working on this today, maybe I can simplify my issue: in Glusterfs 3.0.4, I can create files and directories fine, I can delete files, but not directories. I'm running the server in DEBUG and it's not saying anything. For example, I want to delete /mnt/glusterfs/www/new : [23:12:03] [root at clustr-01 /mnt]# mount -t glusterfs /etc/glusterfs/glusterfs.vol /mnt/glusterfs -o log-level=DEBUG [23:12:13] [root at clustr-01 /mnt]# ls -al /mnt/glusterfs/www/ | grep new drwxrwxrwx 3 www-data www-data 196608 2010-06-18 23:10 new [23:12:26] [root at clustr-01 /mnt]# rm -rf /mnt/glusterfs/www/new/ rm: cannot remove directory `/mnt/glusterfs/www/new/__MACOSX': Transport endpoint is not connected I'm running glusterfsd in another window in DEBUG, and it doesn't log anything when this happens. So I've already deleted the files in that directory, I just can't remove the two remaining directories, new and __MACOSX. Again, I created these yesterday so I haven't made any config changes between then and now, how can I figure out why this is failing? Thanks P On Thu, Jun 17, 2010 at 4:23 PM, phil cryer <phil at cryer.us> wrote: > I'm having problems removing directories, if I do a mv or if I do a rm > I'll get an error like this: > > [00:57:57] [root at clustr-01 /]# rm -rf /mnt/glusterfs/bhl/ > rm: cannot remove directory `/mnt/glusterfs/bhl': Transport endpoint > is not connected > > EdWyse on IRC suggested I run getfattr -m "" on a few bricks, when I > did I got various results (see below). ?Is this a case where I can run > something like backend-cleanup.sh or backend-xattr-sanitize.sh to fix, > or is there a manual command? ?We're around 45TB, so I don't have > anywhere to copy the files off. ?Thanks! > > [16:30:25] [root at clustr-04 /root/bin]# getfattr -m "" /mnt/data04 > getfattr: Removing leading '/' from absolute path names > # file: mnt/data04 > trusted.afr.clustr-04-1 > trusted.afr.clustr-04-10 > trusted.afr.clustr-04-11 > trusted.afr.clustr-04-12 > trusted.afr.clustr-04-13 > trusted.afr.clustr-04-14 > trusted.afr.clustr-04-15 > trusted.afr.clustr-04-16 > trusted.afr.clustr-04-17 > trusted.afr.clustr-04-18 > trusted.afr.clustr-04-19 > trusted.afr.clustr-04-2 > trusted.afr.clustr-04-20 > trusted.afr.clustr-04-21 > trusted.afr.clustr-04-22 > trusted.afr.clustr-04-23 > trusted.afr.clustr-04-24 > trusted.afr.clustr-04-3 > trusted.afr.clustr-04-4 > trusted.afr.clustr-04-5 > trusted.afr.clustr-04-6 > trusted.afr.clustr-04-7 > trusted.afr.clustr-04-8 > trusted.afr.clustr-04-9 > trusted.afr.clustr-05-1 > trusted.afr.clustr-05-10 > trusted.afr.clustr-05-11 > trusted.afr.clustr-05-12 > trusted.afr.clustr-05-13 > trusted.afr.clustr-05-14 > trusted.afr.clustr-05-15 > trusted.afr.clustr-05-16 > trusted.afr.clustr-05-17 > trusted.afr.clustr-05-18 > trusted.afr.clustr-05-19 > trusted.afr.clustr-05-2 > trusted.afr.clustr-05-20 > trusted.afr.clustr-05-21 > trusted.afr.clustr-05-22 > trusted.afr.clustr-05-23 > trusted.afr.clustr-05-24 > trusted.afr.clustr-05-3 > trusted.afr.clustr-05-4 > trusted.afr.clustr-05-5 > trusted.afr.clustr-05-6 > trusted.afr.clustr-05-7 > trusted.afr.clustr-05-8 > trusted.afr.clustr-05-9 > trusted.glusterfs.dht > trusted.posix4.gen > > ----------------------------------------------------------- > Another server > > [01:02:05] [root at clustr-01 /]# ?getfattr -m "" /mnt/data09 > getfattr: Removing leading '/' from absolute path names > # file: mnt/data09 > trusted.afr.clustr-01-10 > trusted.afr.clustr-01-9 > trusted.glusterfs.dht > trusted.glusterfs.test > trusted.posix9.gen > > > [00:43:14] [root at clustr-01 /]# ?getfattr -m "" /mnt/data04 > getfattr: Removing leading '/' from absolute path names > # file: mnt/data04 > trusted.afr.clustr-01-3 > trusted.afr.clustr-01-4 > trusted.glusterfs.dht > trusted.glusterfs.test > trusted.posix4.gen > -- http://philcryer.com