Hi I get the all null pending matrix issue again. This time it is on 3.4.0beta4, and no brick crashed since the volume was created and mounted on the client. Here is the last lines leading to the error on the client. I have many warnings like the one for cbc128.pico before and they seem harmless. The error on cms_env.po is the first message about this file. Since no brick crashed, I just don't understand why it would want to self heal. [2013-06-29 07:48:13.503301] W [client-rpc-fops.c:1538:client3_3_inodelk_cbk] 0-gfs34-client-1: remote operation failed: No such file or directory [2013-06-29 07:48:13.503493] W [client-rpc-fops.c:1538:client3_3_inodelk_cbk] 0-gfs34-client-0: remote operation failed: No such file or directory [2013-06-29 07:48:13.504936] W [client-rpc-fops.c:1538:client3_3_inodelk_cbk] 0-gfs34-client-0: remote operation failed: No such file or directory [2013-06-29 07:48:13.506062] W [client-rpc-fops.c:1538:client3_3_inodelk_cbk] 0-gfs34-client-1: remote operation failed: No such file or directory [2013-06-29 07:48:13.506107] I [afr-lk-common.c:1075:afr_lock_blocking] 0-gfs34-replicate-0: unable to lock on even one child [2013-06-29 07:48:13.506138] I [afr-transaction.c:1063:afr_post_blocking_inodelk_cbk] 0-gfs34-replicate-0: Blocking inodelks failed. [2013-06-29 07:48:13.506283] W [fuse-bridge.c:953:fuse_setattr_cbk] 0-glusterfs-fuse: 15006882: SETATTR() /manu/netbsd/usr/src/crypto/external/bsd/openssl/lib/libcrypto/obj/cbc128.pico => -1 (No such file or directory) [2013-06-29 07:56:05.000389] E [afr-self-heal-common.c:197:afr_sh_print_split_brain_log] 0-gfs34-replicate-0: Unable to self-heal contents of '/manu/netbsd/usr/src/crypto/external/bsd/openssl/lib/libcrypto/obj/cms_env.po' (possible split-brain). Please delete the file from all but the preferred subvolume.- Pending matrix: [ [ 0 0 ] [ 0 0 ] ] [2013-06-29 07:56:05.000949] E [afr-self-heal-common.c:2212:afr_self_heal_completion_cbk] 0-gfs34-replicate-0: background data self-heal failed on /manu/netbsd/usr/src/crypto/external/bsd/openssl/lib/libcrypto/obj/cms_env.po Here are the files on the bricks. The files on bricks 0 and 1 are exactly the same, except that brick 0's version is truncated at 0x01fb4. silo:/export/wd3a -rw-r--r-- 2 manu manu 8116 Jun 29 09:56 /export/wd3a/manu/netbsd/usr/src/crypto/external/bsd/openssl/lib/libcrypto/obj/cms_env.po hangar:/export/wd1a -rw-r--r-- 2 manu manu 9516 Jun 29 09:56 /export/wd1a/manu/netbsd/usr/src/crypto/external/bsd/openssl/lib/libcrypto/obj/cms_env.po hangar:/export/wd3a ls: /export/wd3a/manu/netbsd/usr/src/crypto/external/bsd/openssl/lib/libcrypto/obj/cms_env.po: No such file or directory hotstuff:/export/wd1a ls: /export/wd1a/manu/netbsd/usr/src/crypto/external/bsd/openssl/lib/libcrypto/obj/cms_env.po: No such file or directory -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz manu@xxxxxxxxxx
