Hi there. I'm new to GlusterFS. I'm currently evaluating it for production usage. I have two Storage Servers which use JFS as a filesystem for the underlying export. The setup is supposed to be replicated. I've been experimenting with various settings for benchmarking and such, as well as trying out different failure scenarios. Anyways, the export directory on node 1 is out of sync with node 2. So I mounted the storage volume via glusterfs client on node1 in another directory. The fuse mounted directory is /storage As per Manual I tried doing the "find <gluster-mount> -noleaf -print0 | xargs --null stat >/dev/null" dance, however the logs throw a bunch of errors: ######################################################################### [2011-09-16 18:29:33.759729] E [client3_1-fops.c:1216:client3_1_inodelk_cbk] 0-GLSTORAGE-client-0: error [2011-09-16 18:29:33.759747] I [client3_1-fops.c:1226:client3_1_inodelk_cbk] 0-GLSTORAGE-client-0: remote operation failed: Invalid argument [2011-09-16 18:29:33.759942] E [afr-self-heal-metadata.c:672:afr_sh_metadata_post_nonblocking_inodelk_cbk] 0-GLSTORAGE-replicate-0: Non Blocking metadata inodelks failed for /. [2011-09-16 18:29:33.759961] E [afr-self-heal-metadata.c:674:afr_sh_metadata_post_nonblocking_inodelk_cbk] 0-GLSTORAGE-replicate-0: Metadata self-heal failed for /. [2011-09-16 18:29:33.760167] W [rpc-common.c:64:xdr_to_generic] (-->/usr/local/lib/libgfrpc.so.0(rpc_clnt_notify+0x7d) [0x7f4702a751ad] (-->/usr/local/lib/libgfrpc.so.0(rpc_clnt_handle_reply+0xa5) [0x7f4702a74de5] (-->/usr/local/lib/glusterfs/3.2.3/xlator/protocol/client.so(client3_1_entrylk_cbk+0x52) [0x7f46ff88a572]))) 0-xdr: XDR decoding failed [2011-09-16 18:29:33.760200] E [client3_1-fops.c:1292:client3_1_entrylk_cbk] 0-GLSTORAGE-client-0: error [2011-09-16 18:29:33.760215] I [client3_1-fops.c:1303:client3_1_entrylk_cbk] 0-GLSTORAGE-client-0: remote operation failed: Invalid argument [2011-09-16 18:29:33.760417] E [afr-self-heal-entry.c:2292:afr_sh_post_nonblocking_entry_cbk] 0-GLSTORAGE-replicate-0: Non Blocking entrylks failed for /. [2011-09-16 18:29:33.760447] E [afr-self-heal-common.c:1554:afr_self_heal_completion_cbk] 0-GLSTORAGE-replicate-0: background meta-data entry self-heal failed on / [2011-09-16 18:29:33.760808] I [client3_1-fops.c:2228:client3_1_lookup_cbk] 0-GLSTORAGE-client-0: remote operation failed: Invalid argument ################################################################################### Is this normal? The directory in question already has 150GB of data, so the find command is still running. Will it be ok once it finishes? from what I understand from the manual, the files should repair as the find process runs, or did I misinterpret that? If self heal should fail, is there a failsafe method to ensure that both nodes are in sync again?
