hi All,
I try to remove some directories from our cluster, but ther eis an error:
$ rm -rf _torolni
rm: cannot remove
`_torolni/60_Elements/Prod/T06_APPL/T06_APPL_PACKSHOT/delete/co/cp/T05_APPL_020-co_cp_v001':
Directory not empty
rm: cannot remove
`_torolni/60_Elements/Prod/T06_APPL/T06_APPL_PACKSHOT/T06_APPL_PACKSHOT_010/co/cp/T06_APPL_060-co_cp_v001/f_800x800':
Directory not empty
From the log of the client:
[2014-07-23 17:25:58.532161] W [dht-common.c:419:dht_lookup_dir_cbk]
0-w-vol-dht:
/_torolni/60_Elements/Prod/T06_APPL/T06_APPL_PACKSHOT/delete/co/cp/T05_APPL_020-co_cp_v001:
gfid different on w-vol-client-1
[2014-07-23 17:25:58.532247] W [dht-common.c:419:dht_lookup_dir_cbk]
0-w-vol-dht:
/_torolni/60_Elements/Prod/T06_APPL/T06_APPL_PACKSHOT/delete/co/cp/T05_APPL_020-co_cp_v001:
gfid different on w-vol-client-3
[2014-07-23 17:25:58.532277] W [dht-common.c:419:dht_lookup_dir_cbk]
0-w-vol-dht:
/_torolni/60_Elements/Prod/T06_APPL/T06_APPL_PACKSHOT/delete/co/cp/T05_APPL_020-co_cp_v001:
gfid different on w-vol-client-4
[2014-07-23 17:25:58.532458] W [dht-common.c:419:dht_lookup_dir_cbk]
0-w-vol-dht:
/_torolni/60_Elements/Prod/T06_APPL/T06_APPL_PACKSHOT/delete/co/cp/T05_APPL_020-co_cp_v001:
gfid different on w-vol-client-0
[2014-07-23 17:25:58.533283] W
[client-rpc-fops.c:2541:client3_3_opendir_cbk] 0-w-vol-client-2: remote
operation failed: No such file or directory. Path:
/_torolni/60_Elements/Prod/T06_APPL/T06_APPL_PACKSHOT/delete/co/cp/T05_APPL_020-co_cp_v001
(63cf6edf-f1ba-4657-acd5-c28d225bddf1)
[2014-07-23 17:25:58.543416] W
[client-rpc-fops.c:2541:client3_3_opendir_cbk] 0-w-vol-client-2: remote
operation failed: No such file or directory. Path:
/_torolni/60_Elements/Prod/T06_APPL/T06_APPL_PACKSHOT/delete/co/cp/T05_APPL_020-co_cp_v001
(63cf6edf-f1ba-4657-acd5-c28d225bddf1)
[2014-07-23 17:25:58.557966] W
[client-rpc-fops.c:327:client3_3_mkdir_cbk] 0-w-vol-client-2: remote
operation failed: File exists. Path:
/_torolni/60_Elements/Prod/T06_APPL/T06_APPL_PACKSHOT/delete/co/cp/T05_APPL_020-co_cp_v001
(63cf6edf-f1ba-4657-acd5-c28d225bddf1)
[2014-07-23 17:25:58.653867] W [dht-common.c:419:dht_lookup_dir_cbk]
0-w-vol-dht:
/_torolni/60_Elements/Prod/T06_APPL/T06_APPL_PACKSHOT/T06_APPL_PACKSHOT_010/co/cp/T06_APPL_060-co_cp_v001/f_800x800:
gfid different on w-vol-client-1
[2014-07-23 17:25:58.669396] W
[client-rpc-fops.c:2541:client3_3_opendir_cbk] 0-w-vol-client-1: remote
operation failed: No such file or directory. Path:
/_torolni/60_Elements/Prod/T06_APPL/T06_APPL_PACKSHOT/T06_APPL_PACKSHOT_010/co/cp/T06_APPL_060-co_cp_v001/f_800x800
(6fcb6c71-348f-4e14-bdf4-a0eb5b3df66c)
[2014-07-23 17:25:58.676365] W
[client-rpc-fops.c:2541:client3_3_opendir_cbk] 0-w-vol-client-1: remote
operation failed: No such file or directory. Path:
/_torolni/60_Elements/Prod/T06_APPL/T06_APPL_PACKSHOT/T06_APPL_PACKSHOT_010/co/cp/T06_APPL_060-co_cp_v001/f_800x800
(6fcb6c71-348f-4e14-bdf4-a0eb5b3df66c)
[2014-07-23 17:25:58.677956] W
[client-rpc-fops.c:327:client3_3_mkdir_cbk] 0-w-vol-client-1: remote
operation failed: File exists. Path:
/_torolni/60_Elements/Prod/T06_APPL/T06_APPL_PACKSHOT/T06_APPL_PACKSHOT_010/co/cp/T06_APPL_060-co_cp_v001/f_800x800
(6fcb6c71-348f-4e14-bdf4-a0eb5b3df66c)
[2014-07-23 17:25:58.678121] W
[client-rpc-fops.c:327:client3_3_mkdir_cbk] 0-w-vol-client-3: remote
operation failed: File exists. Path:
/_torolni/60_Elements/Prod/T06_APPL/T06_APPL_PACKSHOT/T06_APPL_PACKSHOT_010/co/cp/T06_APPL_060-co_cp_v001/f_800x800
(6fcb6c71-348f-4e14-bdf4-a0eb5b3df66c)
I found this tool:
https://github.com/vbellur/gfid
Probably it will help.
Why did that happen? Will it happen in the future as well, is this a
persistent bug?
Thanks,
tamas
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-users
