Hi, sorry for direct mailing your emailadress, my mailclient 've problems with this list settings,... No I 've a client log file, additionaly I attached some screenshot's of the virtual machines. With a kvm (virt-manager) machine the problems seem's not so worse. But I prefere virtualbox, here is it worse. Thank You for help, Gregor
[2015-07-20 09:51:41.564106] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.2 (args: /usr/sbin/glusterfs --volfile-server=edgf004.mvz.ffm --volfile-id=/vbstore /import/vbstore) [2015-07-20 09:51:41.754786] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 [2015-07-20 09:51:41.901602] I [event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2 [2015-07-20 09:51:41.902617] I [MSGID: 114020] [client.c:2118:notify] 0-vbstore-client-0: parent translators are ready, attempting connect on transport [2015-07-20 09:51:41.903042] I [MSGID: 114020] [client.c:2118:notify] 0-vbstore-client-1: parent translators are ready, attempting connect on transport [2015-07-20 09:51:41.903689] I [rpc-clnt.c:1819:rpc_clnt_reconfig] 0-vbstore-client-0: changing port to 49152 (from 0) [2015-07-20 09:51:41.904418] I [MSGID: 114057] [client-handshake.c:1438:select_server_supported_programs] 0-vbstore-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2015-07-20 09:51:41.904703] I [MSGID: 114020] [client.c:2118:notify] 0-vbstore-client-2: parent translators are ready, attempting connect on transport [2015-07-20 09:51:41.904872] I [MSGID: 114046] [client-handshake.c:1214:client_setvolume_cbk] 0-vbstore-client-0: Connected to vbstore-client-0, attached to remote volume '/export/vbstore'. [2015-07-20 09:51:41.904908] I [MSGID: 114047] [client-handshake.c:1225:client_setvolume_cbk] 0-vbstore-client-0: Server and Client lk-version numbers are not same, reopening the fds [2015-07-20 09:51:41.904989] I [MSGID: 108005] [afr-common.c:3883:afr_notify] 0-vbstore-replicate-0: Subvolume 'vbstore-client-0' came back up; going online. [2015-07-20 09:51:41.905055] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-vbstore-client-0: Server lk version = 1 Final graph: +------------------------------------------------------------------------------+ 1: volume vbstore-client-0 2: type protocol/client 3: option clnt-lk-version 1 4: option volfile-checksum 0 5: option volfile-key /vbstore 6: option client-version 3.7.2 7: option process-uuid edgf004-5298-2015/07/20-09:51:41:546733-vbstore-client-0-0-0 8: option fops-version 1298437 9: option ping-timeout 42 10: option remote-host edgf004.mvz.ffm 11: option remote-subvolume /export/vbstore 12: option transport-type socket 13: option username 2f90dc67-b1a9-47ef-8dc6-288e3bf0a7cd 14: option password 9db60ebd-b8e9-42b6-a3e3-fa22a38aa646 15: option filter-O_DIRECT enable 16: option send-gids true 17: end-volume 18: 19: volume vbstore-client-1 20: type protocol/client 21: option ping-timeout 42 22: option remote-host edgf005.mvz.ffm 23: option remote-subvolume /export/vbstore 24: option transport-type socket 25: option username 2f90dc67-b1a9-47ef-8dc6-288e3bf0a7cd 26: option password 9db60ebd-b8e9-42b6-a3e3-fa22a38aa646 27: option filter-O_DIRECT enable 28: option send-gids true 29: end-volume 30: 31: volume vbstore-client-2 32: type protocol/client 33: option ping-timeout 42 34: option remote-host edgf006.mvz.ffm 35: option remote-subvolume /export/vbstore 36: option transport-type socket 37: option username 2f90dc67-b1a9-47ef-8dc6-288e3bf0a7cd 38: option password 9db60ebd-b8e9-42b6-a3e3-fa22a38aa646 39: option filter-O_DIRECT enable 40: option send-gids true 41: end-volume 42: [2015-07-20 09:51:41.905505] I [rpc-clnt.c:1819:rpc_clnt_reconfig] 0-vbstore-client-1: changing port to 49152 (from 0) 43: volume vbstore-replicate-0 44: type cluster/replicate 45: option eager-lock enable 46: option quorum-type auto 47: subvolumes vbstore-client-0 vbstore-client-1 vbstore-client-2 48: end-volume 49: 50: volume vbstore-dht 51: type cluster/distribute 52: subvolumes vbstore-replicate-0 53: end-volume 54: 55: volume vbstore-write-behind 56: type performance/write-behind 57: subvolumes vbstore-dht 58: end-volume 59: 60: volume vbstore-readdir-ahead 61: type performance/readdir-ahead 62: subvolumes vbstore-write-behind 63: end-volume 64: 65: volume vbstore-open-behind 66: type performance/open-behind 67: subvolumes vbstore-readdir-ahead 68: end-volume 69: 70: volume vbstore 71: type debug/io-stats 72: option latency-measurement off 73: option count-fop-hits off 74: subvolumes vbstore-open-behind 75: end-volume 76: 77: volume meta-autoload 78: type meta 79: subvolumes vbstore 80: end-volume 81: +------------------------------------------------------------------------------+ [2015-07-20 09:51:41.905974] I [rpc-clnt.c:1819:rpc_clnt_reconfig] 0-vbstore-client-2: changing port to 49153 (from 0) [2015-07-20 09:51:41.906579] I [MSGID: 114057] [client-handshake.c:1438:select_server_supported_programs] 0-vbstore-client-1: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2015-07-20 09:51:41.906885] I [MSGID: 114057] [client-handshake.c:1438:select_server_supported_programs] 0-vbstore-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2015-07-20 09:51:41.906924] I [MSGID: 114046] [client-handshake.c:1214:client_setvolume_cbk] 0-vbstore-client-1: Connected to vbstore-client-1, attached to remote volume '/export/vbstore'. [2015-07-20 09:51:41.906953] I [MSGID: 114047] [client-handshake.c:1225:client_setvolume_cbk] 0-vbstore-client-1: Server and Client lk-version numbers are not same, reopening the fds [2015-07-20 09:51:41.907105] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-vbstore-client-1: Server lk version = 1 [2015-07-20 09:51:41.907252] I [MSGID: 114046] [client-handshake.c:1214:client_setvolume_cbk] 0-vbstore-client-2: Connected to vbstore-client-2, attached to remote volume '/export/vbstore'. [2015-07-20 09:51:41.907274] I [MSGID: 114047] [client-handshake.c:1225:client_setvolume_cbk] 0-vbstore-client-2: Server and Client lk-version numbers are not same, reopening the fds [2015-07-20 09:51:41.911602] I [fuse-bridge.c:5086:fuse_graph_setup] 0-fuse: switched to graph 0 [2015-07-20 09:51:41.911739] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-vbstore-client-2: Server lk version = 1 [2015-07-20 09:51:41.911857] I [fuse-bridge.c:4012:fuse_init] 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.22 kernel 7.23 [2015-07-20 09:51:41.913185] I [afr-common.c:1673:afr_local_discovery_cbk] 0-vbstore-replicate-0: selecting local read_child vbstore-client-0 [2015-07-20 09:54:20.166448] W [socket.c:642:__socket_rwv] 0-vbstore-client-2: readv on 172.22.234.20:49153 failed (Die Wartezeit für die Verbindung ist abgelaufen) [2015-07-20 09:54:20.166506] I [MSGID: 114018] [client.c:2042:client_rpc_notify] 0-vbstore-client-2: disconnected from vbstore-client-2. Client process will keep trying to connect to glusterd until brick's port is available [2015-07-20 09:54:36.594520] E [socket.c:2332:socket_connect_finish] 0-vbstore-client-2: connection to 172.22.234.20:24007 failed (Keine Route zum Zielrechner) [2015-07-20 09:54:53.032987] E [MSGID: 114058] [client-handshake.c:1525:client_query_portmap_cbk] 0-vbstore-client-2: failed to get the port number for remote subvolume. Please run 'gluster volume status' on server to see if brick process is running. [2015-07-20 09:54:57.599751] I [rpc-clnt.c:1819:rpc_clnt_reconfig] 0-vbstore-client-2: changing port to 49153 (from 0) [2015-07-20 09:54:57.600691] I [MSGID: 114057] [client-handshake.c:1438:select_server_supported_programs] 0-vbstore-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2015-07-20 09:54:57.601072] I [MSGID: 114046] [client-handshake.c:1214:client_setvolume_cbk] 0-vbstore-client-2: Connected to vbstore-client-2, attached to remote volume '/export/vbstore'. [2015-07-20 09:54:57.601110] I [MSGID: 114047] [client-handshake.c:1225:client_setvolume_cbk] 0-vbstore-client-2: Server and Client lk-version numbers are not same, reopening the fds [2015-07-20 09:54:57.601399] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-vbstore-client-2: Server lk version = 1 [2015-07-20 09:54:53.033051] I [MSGID: 114018] [client.c:2042:client_rpc_notify] 0-vbstore-client-2: disconnected from vbstore-client-2. Client process will keep trying to connect to glusterd until brick's port is available [2015-07-20 09:56:52.755672] I [afr-common.c:1673:afr_local_discovery_cbk] 0-vbstore-replicate-0: selecting local read_child vbstore-client-0 [2015-07-20 09:59:32.640012] C [rpc-clnt-ping.c:161:rpc_clnt_ping_timer_expired] 0-vbstore-client-2: server 172.22.234.20:49153 has not responded in the last 42 seconds, disconnecting. [2015-07-20 09:59:32.640589] E [rpc-clnt.c:362:saved_frames_unwind] (--> /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(_gf_log_callingfn+0x147)[0x7ff38cdaaed7] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(saved_frames_unwind+0x1cf)[0x7ff38cb77fdf] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7ff38cb780fe] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x9c)[0x7ff38cb798ac] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_notify+0x48)[0x7ff38cb7a058] ))))) 0-vbstore-client-2: forced unwinding frame type(GlusterFS 3.3) op(LOOKUP(27)) called at 2015-07-20 09:58:50.520139 (xid=0x393) [2015-07-20 09:59:32.640635] W [MSGID: 114031] [client-rpc-fops.c:2973:client3_3_lookup_cbk] 0-vbstore-client-2: remote operation failed: Der Socket ist nicht verbunden. Path: / (00000000-0000-0000-0000-000000000001) [Der Socket ist nicht verbunden] [2015-07-20 09:59:32.640888] E [rpc-clnt.c:362:saved_frames_unwind] (--> /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(_gf_log_callingfn+0x147)[0x7ff38cdaaed7] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(saved_frames_unwind+0x1cf)[0x7ff38cb77fdf] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7ff38cb780fe] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x9c)[0x7ff38cb798ac] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_notify+0x48)[0x7ff38cb7a058] ))))) 0-vbstore-client-2: forced unwinding frame type(GF-DUMP) op(NULL(2)) called at 2015-07-20 09:58:50.520146 (xid=0x394) [2015-07-20 09:59:32.640918] W [rpc-clnt-ping.c:204:rpc_clnt_ping_cbk] 0-vbstore-client-2: socket disconnected [2015-07-20 09:59:32.640937] I [MSGID: 114018] [client.c:2042:client_rpc_notify] 0-vbstore-client-2: disconnected from vbstore-client-2. Client process will keep trying to connect to glusterd until brick's port is available [2015-07-20 09:59:32.640977] W [MSGID: 114031] [client-rpc-fops.c:2973:client3_3_lookup_cbk] 0-vbstore-client-2: remote operation failed: Der Socket ist nicht verbunden. Path: /100M (00000000-0000-0000-0000-000000000000) [Der Socket ist nicht verbunden] [2015-07-20 09:59:42.688549] E [MSGID: 114058] [client-handshake.c:1525:client_query_portmap_cbk] 0-vbstore-client-2: failed to get the port number for remote subvolume. Please run 'gluster volume status' on server to see if brick process is running. [2015-07-20 09:59:42.688612] I [MSGID: 114018] [client.c:2042:client_rpc_notify] 0-vbstore-client-2: disconnected from vbstore-client-2. Client process will keep trying to connect to glusterd until brick's port is available [2015-07-20 09:59:48.644891] I [rpc-clnt.c:1819:rpc_clnt_reconfig] 0-vbstore-client-2: changing port to 49153 (from 0) [2015-07-20 09:59:48.645888] I [MSGID: 114057] [client-handshake.c:1438:select_server_supported_programs] 0-vbstore-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2015-07-20 09:59:48.646337] I [MSGID: 114046] [client-handshake.c:1214:client_setvolume_cbk] 0-vbstore-client-2: Connected to vbstore-client-2, attached to remote volume '/export/vbstore'. [2015-07-20 09:59:48.646369] I [MSGID: 114047] [client-handshake.c:1225:client_setvolume_cbk] 0-vbstore-client-2: Server and Client lk-version numbers are not same, reopening the fds [2015-07-20 09:59:48.646568] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-vbstore-client-2: Server lk version = 1 [2015-07-20 10:00:06.089006] I [afr-common.c:1673:afr_local_discovery_cbk] 0-vbstore-replicate-0: selecting local read_child vbstore-client-0 [2015-07-20 10:09:25.482393] I [dht-rename.c:1406:dht_rename] 0-vbstore-dht: renaming /gftest/.gftest.vdi.aoGsNU (hash=vbstore-replicate-0/cache=vbstore-replicate-0) => /gftest/gftest.vdi (hash=vbstore-replicate-0/cache=vbstore-replicate-0) [2015-07-20 10:14:14.087154] I [dht-rename.c:1406:dht_rename] 0-vbstore-dht: renaming /gftest/Logs/VBox.log.2 (hash=vbstore-replicate-0/cache=vbstore-replicate-0) => /gftest/Logs/VBox.log.3 (hash=vbstore-replicate-0/cache=vbstore-replicate-0) [2015-07-20 10:14:14.108711] I [dht-rename.c:1406:dht_rename] 0-vbstore-dht: renaming /gftest/Logs/VBox.log.1 (hash=vbstore-replicate-0/cache=vbstore-replicate-0) => /gftest/Logs/VBox.log.2 (hash=vbstore-replicate-0/cache=<nul>) [2015-07-20 10:14:14.123394] I [dht-rename.c:1406:dht_rename] 0-vbstore-dht: renaming /gftest/Logs/VBox.log (hash=vbstore-replicate-0/cache=vbstore-replicate-0) => /gftest/Logs/VBox.log.1 (hash=vbstore-replicate-0/cache=<nul>) [2015-07-20 10:17:26.980361] I [dht-rename.c:1406:dht_rename] 0-vbstore-dht: renaming /gftest/gftest.vbox (hash=vbstore-replicate-0/cache=vbstore-replicate-0) => /gftest/gftest.vbox-prev (hash=vbstore-replicate-0/cache=vbstore-replicate-0) [2015-07-20 10:17:26.987898] I [dht-rename.c:1406:dht_rename] 0-vbstore-dht: renaming /gftest/gftest.vbox-tmp (hash=vbstore-replicate-0/cache=vbstore-replicate-0) => /gftest/gftest.vbox (hash=vbstore-replicate-0/cache=<nul>) [2015-07-20 10:17:27.314160] I [dht-rename.c:1406:dht_rename] 0-vbstore-dht: renaming /gftest/gftest.vbox (hash=vbstore-replicate-0/cache=vbstore-replicate-0) => /gftest/gftest.vbox-prev (hash=vbstore-replicate-0/cache=vbstore-replicate-0) [2015-07-20 10:17:27.319583] I [dht-rename.c:1406:dht_rename] 0-vbstore-dht: renaming /gftest/gftest.vbox-tmp (hash=vbstore-replicate-0/cache=vbstore-replicate-0) => /gftest/gftest.vbox (hash=vbstore-replicate-0/cache=<nul>) [2015-07-20 10:17:33.726817] I [dht-rename.c:1406:dht_rename] 0-vbstore-dht: renaming /gftest/Logs/VBox.log.2 (hash=vbstore-replicate-0/cache=vbstore-replicate-0) => /gftest/Logs/VBox.log.3 (hash=vbstore-replicate-0/cache=vbstore-replicate-0) [2015-07-20 10:17:33.756906] I [dht-rename.c:1406:dht_rename] 0-vbstore-dht: renaming /gftest/Logs/VBox.log.1 (hash=vbstore-replicate-0/cache=vbstore-replicate-0) => /gftest/Logs/VBox.log.2 (hash=vbstore-replicate-0/cache=<nul>) [2015-07-20 10:17:33.764716] I [dht-rename.c:1406:dht_rename] 0-vbstore-dht: renaming /gftest/Logs/VBox.log (hash=vbstore-replicate-0/cache=vbstore-replicate-0) => /gftest/Logs/VBox.log.1 (hash=vbstore-replicate-0/cache=<nul>) [2015-07-20 10:19:26.918367] W [socket.c:642:__socket_rwv] 0-vbstore-client-2: readv on 172.22.234.20:49153 failed (Die Wartezeit für die Verbindung ist abgelaufen) [2015-07-20 10:19:26.918447] I [MSGID: 114018] [client.c:2042:client_rpc_notify] 0-vbstore-client-2: disconnected from vbstore-client-2. Client process will keep trying to connect to glusterd until brick's port is available [2015-07-20 10:20:10.700414] E [MSGID: 114058] [client-handshake.c:1525:client_query_portmap_cbk] 0-vbstore-client-2: failed to get the port number for remote subvolume. Please run 'gluster volume status' on server to see if brick process is running. [2015-07-20 10:20:10.700488] I [MSGID: 114018] [client.c:2042:client_rpc_notify] 0-vbstore-client-2: disconnected from vbstore-client-2. Client process will keep trying to connect to glusterd until brick's port is available [2015-07-20 10:20:13.852789] I [rpc-clnt.c:1819:rpc_clnt_reconfig] 0-vbstore-client-2: changing port to 49153 (from 0) [2015-07-20 10:20:13.853755] I [MSGID: 114057] [client-handshake.c:1438:select_server_supported_programs] 0-vbstore-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2015-07-20 10:20:13.854120] I [MSGID: 114046] [client-handshake.c:1214:client_setvolume_cbk] 0-vbstore-client-2: Connected to vbstore-client-2, attached to remote volume '/export/vbstore'. [2015-07-20 10:20:13.854149] I [MSGID: 114047] [client-handshake.c:1225:client_setvolume_cbk] 0-vbstore-client-2: Server and Client lk-version numbers are not same, reopening the fds [2015-07-20 10:20:13.854180] I [MSGID: 114042] [client-handshake.c:1056:client_post_handshake] 0-vbstore-client-2: 2 fds open - Delaying child_up until they are re-opened [2015-07-20 10:20:13.855639] I [MSGID: 114041] [client-handshake.c:678:client_child_up_reopen_done] 0-vbstore-client-2: last fd open'd/lock-self-heal'd - notifying CHILD-UP [2015-07-20 10:20:13.855829] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-vbstore-client-2: Server lk version = 1 [2015-07-20 10:23:13.885624] C [rpc-clnt-ping.c:161:rpc_clnt_ping_timer_expired] 0-vbstore-client-2: server 172.22.234.20:49153 has not responded in the last 42 seconds, disconnecting. [2015-07-20 10:23:13.886098] E [rpc-clnt.c:362:saved_frames_unwind] (--> /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(_gf_log_callingfn+0x147)[0x7ff38cdaaed7] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(saved_frames_unwind+0x1cf)[0x7ff38cb77fdf] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7ff38cb780fe] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x9c)[0x7ff38cb798ac] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_notify+0x48)[0x7ff38cb7a058] ))))) 0-vbstore-client-2: forced unwinding frame type(GlusterFS 3.3) op(FINODELK(30)) called at 2015-07-20 10:22:31.394224 (xid=0x17816) [2015-07-20 10:23:13.886141] E [MSGID: 114031] [client-rpc-fops.c:1675:client3_3_finodelk_cbk] 0-vbstore-client-2: remote operation failed: Der Socket ist nicht verbunden [Der Socket ist nicht verbunden] [2015-07-20 10:23:13.886421] E [rpc-clnt.c:362:saved_frames_unwind] (--> /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(_gf_log_callingfn+0x147)[0x7ff38cdaaed7] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(saved_frames_unwind+0x1cf)[0x7ff38cb77fdf] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7ff38cb780fe] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x9c)[0x7ff38cb798ac] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_notify+0x48)[0x7ff38cb7a058] ))))) 0-vbstore-client-2: forced unwinding frame type(GlusterFS 3.3) op(FINODELK(30)) called at 2015-07-20 10:22:31.394556 (xid=0x17818) [2015-07-20 10:23:13.886648] E [rpc-clnt.c:362:saved_frames_unwind] (--> /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(_gf_log_callingfn+0x147)[0x7ff38cdaaed7] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(saved_frames_unwind+0x1cf)[0x7ff38cb77fdf] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7ff38cb780fe] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x9c)[0x7ff38cb798ac] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_notify+0x48)[0x7ff38cb7a058] ))))) 0-vbstore-client-2: forced unwinding frame type(GlusterFS 3.3) op(FINODELK(30)) called at 2015-07-20 10:22:31.394745 (xid=0x17819) [2015-07-20 10:23:13.886875] E [rpc-clnt.c:362:saved_frames_unwind] (--> /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(_gf_log_callingfn+0x147)[0x7ff38cdaaed7] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(saved_frames_unwind+0x1cf)[0x7ff38cb77fdf] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7ff38cb780fe] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x9c)[0x7ff38cb798ac] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_notify+0x48)[0x7ff38cb7a058] ))))) 0-vbstore-client-2: forced unwinding frame type(GlusterFS 3.3) op(FINODELK(30)) called at 2015-07-20 10:22:31.395030 (xid=0x1781a) [2015-07-20 10:23:13.887100] E [rpc-clnt.c:362:saved_frames_unwind] (--> /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(_gf_log_callingfn+0x147)[0x7ff38cdaaed7] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(saved_frames_unwind+0x1cf)[0x7ff38cb77fdf] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7ff38cb780fe] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x9c)[0x7ff38cb798ac] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_notify+0x48)[0x7ff38cb7a058] ))))) 0-vbstore-client-2: forced unwinding frame type(GlusterFS 3.3) op(FINODELK(30)) called at 2015-07-20 10:22:31.395222 (xid=0x1781b) [2015-07-20 10:23:13.887330] E [rpc-clnt.c:362:saved_frames_unwind] (--> /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(_gf_log_callingfn+0x147)[0x7ff38cdaaed7] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(saved_frames_unwind+0x1cf)[0x7ff38cb77fdf] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7ff38cb780fe] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x9c)[0x7ff38cb798ac] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_notify+0x48)[0x7ff38cb7a058] ))))) 0-vbstore-client-2: forced unwinding frame type(GlusterFS 3.3) op(FINODELK(30)) called at 2015-07-20 10:22:31.395285 (xid=0x1781c) [2015-07-20 10:23:13.887505] I [socket.c:3358:socket_submit_request] 0-vbstore-client-2: not connected (priv->connected = 0) [2015-07-20 10:23:13.887532] W [rpc-clnt.c:1571:rpc_clnt_submit] 0-vbstore-client-2: failed to submit rpc-request (XID: 0x1781e Program: GlusterFS 3.3, ProgVers: 330, Proc: 30) to rpc-transport (vbstore-client-2) [2015-07-20 10:23:13.887578] E [rpc-clnt.c:362:saved_frames_unwind] (--> /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(_gf_log_callingfn+0x147)[0x7ff38cdaaed7] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(saved_frames_unwind+0x1cf)[0x7ff38cb77fdf] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7ff38cb780fe] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x9c)[0x7ff38cb798ac] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_notify+0x48)[0x7ff38cb7a058] ))))) 0-vbstore-client-2: forced unwinding frame type(GlusterFS 3.3) op(FINODELK(30)) called at 2015-07-20 10:23:01.840094 (xid=0x1781d) [2015-07-20 10:23:13.887723] W [rpc-clnt.c:1571:rpc_clnt_submit] 0-vbstore-client-2: failed to submit rpc-request (XID: 0x1781f Program: GlusterFS 3.3, ProgVers: 330, Proc: 30) to rpc-transport (vbstore-client-2) [2015-07-20 10:23:13.887826] E [rpc-clnt.c:362:saved_frames_unwind] (--> /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(_gf_log_callingfn+0x147)[0x7ff38cdaaed7] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(saved_frames_unwind+0x1cf)[0x7ff38cb77fdf] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7ff38cb780fe] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x9c)[0x7ff38cb798ac] (--> /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_notify+0x48)[0x7ff38cb7a058] ))))) 0-vbstore-client-2: forced unwinding frame type(GF-DUMP) op(NULL(2)) called at 2015-07-20 10:22:31.394230 (xid=0x17817) [2015-07-20 10:23:13.887855] W [rpc-clnt-ping.c:204:rpc_clnt_ping_cbk] 0-vbstore-client-2: socket disconnected [2015-07-20 10:23:13.887873] I [MSGID: 114018] [client.c:2042:client_rpc_notify] 0-vbstore-client-2: disconnected from vbstore-client-2. Client process will keep trying to connect to glusterd until brick's port is available The message "E [MSGID: 114031] [client-rpc-fops.c:1675:client3_3_finodelk_cbk] 0-vbstore-client-2: remote operation failed: Der Socket ist nicht verbunden [Der Socket ist nicht verbunden]" repeated 13 times between [2015-07-20 10:23:13.886141] and [2015-07-20 10:23:13.888721] [2015-07-20 10:23:44.946851] I [rpc-clnt.c:1819:rpc_clnt_reconfig] 0-vbstore-client-2: changing port to 49153 (from 0) [2015-07-20 10:23:44.948039] I [MSGID: 114057] [client-handshake.c:1438:select_server_supported_programs] 0-vbstore-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2015-07-20 10:23:44.948444] I [MSGID: 114046] [client-handshake.c:1214:client_setvolume_cbk] 0-vbstore-client-2: Connected to vbstore-client-2, attached to remote volume '/export/vbstore'. [2015-07-20 10:23:44.948474] I [MSGID: 114047] [client-handshake.c:1225:client_setvolume_cbk] 0-vbstore-client-2: Server and Client lk-version numbers are not same, reopening the fds [2015-07-20 10:23:44.948490] I [MSGID: 114042] [client-handshake.c:1056:client_post_handshake] 0-vbstore-client-2: 2 fds open - Delaying child_up until they are re-opened [2015-07-20 10:23:44.950156] I [MSGID: 114041] [client-handshake.c:678:client_child_up_reopen_done] 0-vbstore-client-2: last fd open'd/lock-self-heal'd - notifying CHILD-UP [2015-07-20 10:23:44.950432] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-vbstore-client-2: Server lk version = 1 [2015-07-20 10:28:45.332617] I [afr-self-heal-common.c:651:afr_log_selfheal] 0-vbstore-replicate-0: Completed data selfheal on 4932948d-8503-4e12-8d32-92ecb5a0c513. source=0 sinks=2 [2015-07-20 10:42:37.148658] I [afr-common.c:1673:afr_local_discovery_cbk] 0-vbstore-replicate-0: selecting local read_child vbstore-client-0 [2015-07-20 10:42:41.265433] I [dht-rename.c:1406:dht_rename] 0-vbstore-dht: renaming /gftest/gftest.vbox (hash=vbstore-replicate-0/cache=vbstore-replicate-0) => /gftest/gftest.vbox-prev (hash=vbstore-replicate-0/cache=vbstore-replicate-0) [2015-07-20 10:42:41.284706] I [dht-rename.c:1406:dht_rename] 0-vbstore-dht: renaming /gftest/gftest.vbox-tmp (hash=vbstore-replicate-0/cache=vbstore-replicate-0) => /gftest/gftest.vbox (hash=vbstore-replicate-0/cache=<nul>) [2015-07-20 10:42:41.537759] I [dht-rename.c:1406:dht_rename] 0-vbstore-dht: renaming /gftest/gftest.vbox (hash=vbstore-replicate-0/cache=vbstore-replicate-0) => /gftest/gftest.vbox-prev (hash=vbstore-replicate-0/cache=vbstore-replicate-0) [2015-07-20 10:42:41.547163] I [dht-rename.c:1406:dht_rename] 0-vbstore-dht: renaming /gftest/gftest.vbox-tmp (hash=vbstore-replicate-0/cache=vbstore-replicate-0) => /gftest/gftest.vbox (hash=vbstore-replicate-0/cache=<nul>) [2015-07-20 11:06:58.545056] I [dht-rename.c:1406:dht_rename] 0-vbstore-dht: renaming /gftest/Logs/VBox.log.2 (hash=vbstore-replicate-0/cache=vbstore-replicate-0) => /gftest/Logs/VBox.log.3 (hash=vbstore-replicate-0/cache=vbstore-replicate-0) [2015-07-20 11:06:58.614076] I [dht-rename.c:1406:dht_rename] 0-vbstore-dht: renaming /gftest/Logs/VBox.log.1 (hash=vbstore-replicate-0/cache=vbstore-replicate-0) => /gftest/Logs/VBox.log.2 (hash=vbstore-replicate-0/cache=<nul>) [2015-07-20 11:06:58.621953] I [dht-rename.c:1406:dht_rename] 0-vbstore-dht: renaming /gftest/Logs/VBox.log (hash=vbstore-replicate-0/cache=vbstore-replicate-0) => /gftest/Logs/VBox.log.1 (hash=vbstore-replicate-0/cache=<nul>) [2015-07-20 11:07:59.597842] I [dht-rename.c:1406:dht_rename] 0-vbstore-dht: renaming /gftest/gftest.vbox (hash=vbstore-replicate-0/cache=vbstore-replicate-0) => /gftest/gftest.vbox-prev (hash=vbstore-replicate-0/cache=vbstore-replicate-0) [2015-07-20 11:07:59.603347] I [dht-rename.c:1406:dht_rename] 0-vbstore-dht: renaming /gftest/gftest.vbox-tmp (hash=vbstore-replicate-0/cache=vbstore-replicate-0) => /gftest/gftest.vbox (hash=vbstore-replicate-0/cache=<nul>)
Attachment:
output_onhealKVM.png
Description: PNG image
Attachment:
output_onhealVirtualbox1.png
Description: PNG image
Attachment:
output_onhealVirtualbox2.png
Description: PNG image
_______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-users
