Hello List, i am running ovirt (CentOS) on top of glusterfs. I have a 3 Node replica. Versions see below. Looks like i can not get my node1 (v 3.7.8) together with the othet two (v3.7.0). The error i get when i try to " mount -t glusterfs 10.10.3.7:/RaidVolC /mnt/": [2016-02-20 10:27:30.890701] W [socket.c:869:__socket_keepalive] 0-socket: failed to set TCP_USER_TIMEOUT -1000 on socket 14, Invalid argument [2016-02-20 10:27:30.890728] E [socket.c:2965:socket_connect] 0-management: Failed to set keep-alive: Invalid argument [2016-02-20 10:27:30.891296] W [socket.c:588:__socket_rwv] 0-management: readv on 10.10.3.7:24007 failed (No data available) [2016-02-20 10:27:30.891671] E [rpc-clnt.c:362:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x192)[0x7ff82c50bab2] (--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7ff82c2d68de] (--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7ff82c2d69ee] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x7a)[0x7ff82c2d837a] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x88)[0x7ff82c2d8ba8] ))))) 0-management: forced unwinding frame type(GLUSTERD-DUMP) op(DUMP(1)) called at 2016-02-20 10:27:30.891063 (xid=0x35) The message "W [MSGID: 106118] [glusterd-handler.c:5149:__glusterd_peer_rpc_notify] 0-management: Lock not released for RaidVolC" repeated 3 times between [2016-02-20 10:27:24.873207] and [2016-02-20 10:27:27.886916] [2016-02-20 10:27:30.891704] E [MSGID: 106167] [glusterd-handshake.c:2074:__glusterd_peer_dump_version_cbk] 0-management: Error through RPC layer, retry again later [2016-02-20 10:27:30.891871] W [glusterd-locks.c:681:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/3.7.8/xlator/mgmt/glusterd.so(glusterd_big_locked_notify+0x4c) [0x7ff821062b9c] -->/usr/lib64/glusterfs/3.7.8/xlator/mgmt/glusterd.so(__glusterd_peer_rpc_notify+0x162) [0x7ff82106ce72] -->/usr/lib64/glusterfs/3.7.8/xlator/mgmt/glusterd.so(glusterd_mgmt_v3_unlock+0x58a) [0x7ff82110c73a] ) 0-management: Lock for vol RaidVolB not held [2016-02-20 10:27:30.892001] W [glusterd-locks.c:681:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/3.7.8/xlator/mgmt/glusterd.so(glusterd_big_locked_notify+0x4c) [0x7ff821062b9c] -->/usr/lib64/glusterfs/3.7.8/xlator/mgmt/glusterd.so(__glusterd_peer_rpc_notify+0x162) [0x7ff82106ce72] -->/usr/lib64/glusterfs/3.7.8/xlator/mgmt/glusterd.so(glusterd_mgmt_v3_unlock+0x58a) [0x7ff82110c73a] ) 0-management: Lock for vol RaidVolC not held The message "W [MSGID: 106118] [glusterd-handler.c:5149:__glusterd_peer_rpc_notify] 0-management: Lock not released for RaidVolB" repeated 3 times between [2016-02-20 10:27:24.877923] and [2016-02-20 10:27:30.891888] [2016-02-20 10:27:30.892023] W [MSGID: 106118] [glusterd-handler.c:5149:__glusterd_peer_rpc_notify] 0-management: Lock not released for RaidVolC [2016-02-20 10:27:30.895617] W [socket.c:869:__socket_keepalive] 0-socket: failed to set TCP_USER_TIMEOUT -1000 on socket 14, Invalid argument [2016-02-20 10:27:30.895641] E [socket.c:2965:socket_connect] 0-management: Failed to set keep-alive: Invalid argument [2016-02-20 10:27:30.896300] W [socket.c:588:__socket_rwv] 0-management: readv on 10.10.1.6:24007 failed (No data available) [2016-02-20 10:27:30.896541] E [rpc-clnt.c:362:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x192)[0x7ff82c50bab2] (--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7ff82c2d68de] (--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7ff82c2d69ee] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x7a)[0x7ff82c2d837a] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x88)[0x7ff82c2d8ba8] ))))) 0-management: forced unwinding frame type(GLUSTERD-DUMP) op(DUMP(1)) called at 2016-02-20 10:27:30.895995 (xid=0x35) [2016-02-20 10:27:30.896703] W [glusterd-locks.c:681:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/3.7.8/xlator/mgmt/glusterd.so(glusterd_big_locked_notify+0x4c) [0x7ff821062b9c] -->/usr/lib64/glusterfs/3.7.8/xlator/mgmt/glusterd.so(__glusterd_peer_rpc_notify+0x162) [0x7ff82106ce72] -->/usr/lib64/glusterfs/3.7.8/xlator/mgmt/glusterd.so(glusterd_mgmt_v3_unlock+0x58a) [0x7ff82110c73a] ) 0-management: Lock for vol RaidVolB not held [2016-02-20 10:27:30.896584] I [MSGID: 106004] [glusterd-handler.c:5127:__glusterd_peer_rpc_notify] 0-management: Peer <ovirt-node06-stgt.stuttgart.imos.net> (<08884518-2db7-4429-ab2f-019d03a02b76>), in state <Peer in Cluster>, has disconnected from glusterd. [2016-02-20 10:27:30.896720] W [MSGID: 106118] [glusterd-handler.c:5149:__glusterd_peer_rpc_notify] 0-management: Lock not released for RaidVolB [2016-02-20 10:27:30.896854] W [glusterd-locks.c:681:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/3.7.8/xlator/mgmt/glusterd.so(glusterd_big_locked_notify+0x4c) [0x7ff821062b9c] -->/usr/lib64/glusterfs/3.7.8/xlator/mgmt/glusterd.so(__glusterd_peer_rpc_notify+0x162) [0x7ff82106ce72] -->/usr/lib64/glusterfs/3.7.8/xlator/mgmt/glusterd.so(glusterd_mgmt_v3_unlock+0x58a) [0x7ff82110c73a] ) 0-management: Lock for vol RaidVolC not held Any idea what the problem is? I had a network problem which is sloved now. But now i am stuck with this. Node1: ============ rpm -qa |grep gluster glusterfs-fuse-3.7.8-1.el7.x86_64 glusterfs-3.7.8-1.el7.x86_64 glusterfs-cli-3.7.8-1.el7.x86_64 glusterfs-client-xlators-3.7.8-1.el7.x86_64 glusterfs-rdma-3.7.8-1.el7.x86_64 vdsm-gluster-4.16.30-0.el7.centos.noarch glusterfs-api-3.7.8-1.el7.x86_64 glusterfs-libs-3.7.8-1.el7.x86_64 glusterfs-server-3.7.8-1.el7.x86_64 Node2: ============== rpm -qa |grep gluster glusterfs-fuse-3.7.0-1.el7.x86_64 glusterfs-libs-3.7.0-1.el7.x86_64 glusterfs-api-3.7.0-1.el7.x86_64 glusterfs-cli-3.7.0-1.el7.x86_64 glusterfs-server-3.7.0-1.el7.x86_64 glusterfs-3.7.0-1.el7.x86_64 glusterfs-rdma-3.7.0-1.el7.x86_64 vdsm-gluster-4.16.14-0.el7.noarch glusterfs-client-xlators-3.7.0-1.el7.x86_64 Node3: ================= rpm -qa|grep glus glusterfs-3.7.0-1.el7.x86_64 glusterfs-rdma-3.7.0-1.el7.x86_64 glusterfs-client-xlators-3.7.0-1.el7.x86_64 glusterfs-libs-3.7.0-1.el7.x86_64 glusterfs-api-3.7.0-1.el7.x86_64 glusterfs-cli-3.7.0-1.el7.x86_64 glusterfs-server-3.7.0-1.el7.x86_64 vdsm-gluster-4.16.14-0.el7.noarch glusterfs-fuse-3.7.0-1.el7.x86_64 Thanks, Mario _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-users
