Hi All, since we need to restore gluster storage as soon as possible, we decided to ignore the few files that could be lost and to go ahead. So we cleaned all bricks content of servers s04, s05 and s06. As planned some days ago, we executed the following commands: gluster peer detach s04 gluster peer detach s05 gluster peer detach s06 gluster peer probe s04 gluster peer probe s05 gluster peer probe s06 gluster volume add-brick tier2 s04-stg:/gluster/mnt1/brick s05-stg:/gluster/mnt1/brick s06-stg:/gluster/mnt1/brick s04-stg:/gluster/mnt2/brick s05-stg:/gluster/mnt2/brick s06-stg:/gluster/mnt2/brick s04-stg:/gluster/mnt3/brick s05-stg:/gluster/mnt3/brick s06-stg:/gluster/mnt3/brick s04-stg:/gluster/mnt4/brick s05-stg:/gluster/mnt4/brick s06-stg:/gluster/mnt4/brick s04-stg:/gluster/mnt5/brick s05-stg:/gluster/mnt5/brick s06-stg:/gluster/mnt5/brick s04-stg:/gluster/mnt6/brick s05-stg:/gluster/mnt6/brick s06-stg:/gluster/mnt6/brick s04-stg:/gluster/mnt7/brick s05-stg:/gluster/mnt7/brick s06-stg:/gluster/mnt7/brick s04-stg:/gluster/mnt8/brick s05-stg:/gluster/mnt8/brick s06-stg:/gluster/mnt8/brick s04-stg:/gluster/mnt9/brick s05-stg:/gluster/mnt9/brick s06-stg:/gluster/mnt9/brick s04-stg:/gluster/mnt10/brick s05-stg:/gluster/mnt10/brick s06-stg:/gluster/mnt10/brick s04-stg:/gluster/mnt11/brick s05-stg:/gluster/mnt11/brick s06-stg:/gluster/mnt11/brick s04-stg:/gluster/mnt12/brick s05-stg:/gluster/mnt12/brick s06-stg:/gluster/mnt12/brick force gluster volume rebalance tier2 fix-layout start Everything seem to be fine and fix-layout ended. [root@s01 ~]# gluster volume rebalance tier2 status Node status run time in h:m:s --------- ----------- ------------ localhost fix-layout completed 12:11:6 s02-stg fix-layout completed 12:11:18 s03-stg fix-layout completed 12:11:12 s04-stg fix-layout completed 12:11:20 s05-stg fix-layout completed 12:11:14 s06-stg fix-layout completed 12:10:47 volume rebalance: tier2: success [root@s01 ~]# gluster volume info Volume Name: tier2 Type: Distributed-Disperse Volume ID: a28d88c5-3295-4e35-98d4-210b3af9358c Status: Started Snapshot Count: 0 Number of Bricks: 12 x (4 + 2) = 72 Transport-type: tcp Bricks: Brick1: s01-stg:/gluster/mnt1/brick Brick2: s02-stg:/gluster/mnt1/brick Brick3: s03-stg:/gluster/mnt1/brick Brick4: s01-stg:/gluster/mnt2/brick Brick5: s02-stg:/gluster/mnt2/brick Brick6: s03-stg:/gluster/mnt2/brick Brick7: s01-stg:/gluster/mnt3/brick Brick8: s02-stg:/gluster/mnt3/brick Brick9: s03-stg:/gluster/mnt3/brick Brick10: s01-stg:/gluster/mnt4/brick Brick11: s02-stg:/gluster/mnt4/brick Brick12: s03-stg:/gluster/mnt4/brick Brick13: s01-stg:/gluster/mnt5/brick Brick14: s02-stg:/gluster/mnt5/brick Brick15: s03-stg:/gluster/mnt5/brick Brick16: s01-stg:/gluster/mnt6/brick Brick17: s02-stg:/gluster/mnt6/brick Brick18: s03-stg:/gluster/mnt6/brick Brick19: s01-stg:/gluster/mnt7/brick Brick20: s02-stg:/gluster/mnt7/brick Brick21: s03-stg:/gluster/mnt7/brick Brick22: s01-stg:/gluster/mnt8/brick Brick23: s02-stg:/gluster/mnt8/brick Brick24: s03-stg:/gluster/mnt8/brick Brick25: s01-stg:/gluster/mnt9/brick Brick26: s02-stg:/gluster/mnt9/brick Brick27: s03-stg:/gluster/mnt9/brick Brick28: s01-stg:/gluster/mnt10/brick Brick29: s02-stg:/gluster/mnt10/brick Brick30: s03-stg:/gluster/mnt10/brick Brick31: s01-stg:/gluster/mnt11/brick Brick32: s02-stg:/gluster/mnt11/brick Brick33: s03-stg:/gluster/mnt11/brick Brick34: s01-stg:/gluster/mnt12/brick Brick35: s02-stg:/gluster/mnt12/brick Brick36: s03-stg:/gluster/mnt12/brick Brick37: s04-stg:/gluster/mnt1/brick Brick38: s05-stg:/gluster/mnt1/brick Brick39: s06-stg:/gluster/mnt1/brick Brick40: s04-stg:/gluster/mnt2/brick Brick41: s05-stg:/gluster/mnt2/brick Brick42: s06-stg:/gluster/mnt2/brick Brick43: s04-stg:/gluster/mnt3/brick Brick44: s05-stg:/gluster/mnt3/brick Brick45: s06-stg:/gluster/mnt3/brick Brick46: s04-stg:/gluster/mnt4/brick Brick47: s05-stg:/gluster/mnt4/brick Brick48: s06-stg:/gluster/mnt4/brick Brick49: s04-stg:/gluster/mnt5/brick Brick50: s05-stg:/gluster/mnt5/brick Brick51: s06-stg:/gluster/mnt5/brick Brick52: s04-stg:/gluster/mnt6/brick Brick53: s05-stg:/gluster/mnt6/brick Brick54: s06-stg:/gluster/mnt6/brick Brick55: s04-stg:/gluster/mnt7/brick Brick56: s05-stg:/gluster/mnt7/brick Brick57: s06-stg:/gluster/mnt7/brick Brick58: s04-stg:/gluster/mnt8/brick Brick59: s05-stg:/gluster/mnt8/brick Brick60: s06-stg:/gluster/mnt8/brick Brick61: s04-stg:/gluster/mnt9/brick Brick62: s05-stg:/gluster/mnt9/brick Brick63: s06-stg:/gluster/mnt9/brick Brick64: s04-stg:/gluster/mnt10/brick Brick65: s05-stg:/gluster/mnt10/brick Brick66: s06-stg:/gluster/mnt10/brick Brick67: s04-stg:/gluster/mnt11/brick Brick68: s05-stg:/gluster/mnt11/brick Brick69: s06-stg:/gluster/mnt11/brick Brick70: s04-stg:/gluster/mnt12/brick Brick71: s05-stg:/gluster/mnt12/brick Brick72: s06-stg:/gluster/mnt12/brick Options Reconfigured: network.ping-timeout: 42 features.scrub: Active features.bitrot: on features.inode-quota: on features.quota: on performance.client-io-threads: on cluster.min-free-disk: 10 cluster.quorum-type: none transport.address-family: inet nfs.disable: on server.event-threads: 4 client.event-threads: 4 cluster.lookup-optimize: on performance.readdir-ahead: on performance.parallel-readdir: off cluster.readdir-optimize: on features.cache-invalidation: on features.cache-invalidation-timeout: 600 performance.stat-prefetch: on performance.cache-invalidation: on performance.md-cache-timeout: 600 network.inode-lru-limit: 50000 performance.io-cache: off disperse.cpu-extensions: auto performance.io-thread-count: 16 features.quota-deem-statfs: on features.default-soft-limit: 90 cluster.server-quorum-type: none diagnostics.latency-measurement: on diagnostics.count-fop-hits: on cluster.brick-multiplex: off cluster.server-quorum-ratio: 51% The last step should be the data rebalance between the servers, but rebalance failed soon with a lot of errors like the following ones: [2018-10-05 23:48:38.644978] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-70: Server lk version = 1 [2018-10-05 23:48:44.735323] I [dht-rebalance.c:4512:gf_defrag_start_crawl] 0-tier2-dht: gf_defrag_start_crawl using commit hash 3720331860 [2018-10-05 23:48:44.736205] W [MSGID: 122040] [ec-common.c:1097:ec_prepare_update_cbk] 0-tier2-disperse-7: Failed to get size and version [Input/output error] [2018-10-05 23:48:44.736266] E [MSGID: 122034] [ec-common.c:613:ec_child_select] 0-tier2-disperse-7: Insufficient available children for this request (have 0, need 4) [2018-10-05 23:48:44.736282] E [MSGID: 122037] [ec-common.c:2040:ec_update_size_version_done] 0-tier2-disperse-7: Failed to update version and size [Input/output error] [2018-10-05 23:48:44.736377] W [MSGID: 122040] [ec-common.c:1097:ec_prepare_update_cbk] 0-tier2-disperse-8: Failed to get size and version [Input/output error] [2018-10-05 23:48:44.736436] E [MSGID: 122034] [ec-common.c:613:ec_child_select] 0-tier2-disperse-8: Insufficient available children for this request (have 0, need 4) [2018-10-05 23:48:44.736459] E [MSGID: 122037] [ec-common.c:2040:ec_update_size_version_done] 0-tier2-disperse-8: Failed to update version and size [Input/output error] [2018-10-05 23:48:44.736460] W [MSGID: 122040] [ec-common.c:1097:ec_prepare_update_cbk] 0-tier2-disperse-10: Failed to get size and version [Input/output error] [2018-10-05 23:48:44.736537] W [MSGID: 122040] [ec-common.c:1097:ec_prepare_update_cbk] 0-tier2-disperse-9: Failed to get size and version [Input/output error] [2018-10-05 23:48:44.736571] E [MSGID: 122034] [ec-common.c:613:ec_child_select] 0-tier2-disperse-10: Insufficient available children for this request (have 0, need 4) [2018-10-05 23:48:44.736574] E [MSGID: 122034] [ec-common.c:613:ec_child_select] 0-tier2-disperse-9: Insufficient available children for this request (have 0, need 4) [2018-10-05 23:48:44.736604] E [MSGID: 122037] [ec-common.c:2040:ec_update_size_version_done] 0-tier2-disperse-9: Failed to update version and size [Input/output error] [2018-10-05 23:48:44.736604] E [MSGID: 122037] [ec-common.c:2040:ec_update_size_version_done] 0-tier2-disperse-10: Failed to update version and size [Input/output error] [2018-10-05 23:48:44.736827] W [MSGID: 122040] [ec-common.c:1097:ec_prepare_update_cbk] 0-tier2-disperse-11: Failed to get size and version [Input/output error] [2018-10-05 23:48:44.736887] E [MSGID: 122034] [ec-common.c:613:ec_child_select] 0-tier2-disperse-11: Insufficient available children for this request (have 0, need 4) [2018-10-05 23:48:44.736904] E [MSGID: 122037] [ec-common.c:2040:ec_update_size_version_done] 0-tier2-disperse-11: Failed to update version and size [Input/output error] [2018-10-05 23:48:44.740337] W [MSGID: 122040] [ec-common.c:1097:ec_prepare_update_cbk] 0-tier2-disperse-6: Failed to get size and version [Input/output error] [2018-10-05 23:48:44.740381] E [MSGID: 122034] [ec-common.c:613:ec_child_select] 0-tier2-disperse-6: Insufficient available children for this request (have 0, need 4) [2018-10-05 23:48:44.740394] E [MSGID: 122037] [ec-common.c:2040:ec_update_size_version_done] 0-tier2-disperse-6: Failed to update version and size [Input/output error] [2018-10-05 23:48:50.066103] I [MSGID: 109081] [dht-common.c:4379:dht_setxattr] 0-tier2-dht: fixing the layout of / In attachment you can find the first logs captured during the rebalance execution. In your opinion, is there a way to restore the gluster storage or all the data have been lost? Thank you in advance, Mauro |
[2018-10-05 23:48:26.048453] I [MSGID: 100030] [glusterfsd.c:2511:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.12.14 (args: /usr/sbin/glusterfs -s localhost --volfile-id rebalance/tier2 --xlator-option *dht.use-readdirp=yes --xlator-option *dht.lookup-unhashed=yes --xlator-option *dht.assert-no-child-down=yes --xlator-option *replicate*.data-self-heal=off --xlator-option *replicate*.metadata-self-heal=off --xlator-option *replicate*.entry-self-heal=off --xlator-option *dht.readdir-optimize=on --xlator-option *dht.rebalance-cmd=1 --xlator-option *dht.node-uuid=b734b083-4630-4523-9402-05d03565efee --xlator-option *dht.commit-hash=3720331860 --socket-file /var/run/gluster/gluster-rebalance-a28d88c5-3295-4e35-98d4-210b3af9358c.sock --pid-file /var/lib/glusterd/vols/tier2/rebalance/b734b083-4630-4523-9402-05d03565efee.pid -l /var/log/glusterfs/tier2-rebalance.log) [2018-10-05 23:48:26.053751] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-glusterfs: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction [2018-10-05 23:48:26.058160] I [MSGID: 101190] [event-epoll.c:613:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 [2018-10-05 23:48:31.055738] I [MSGID: 109104] [dht-shared.c:690:dht_init] 0-tier2-dht: dht_init using commit hash 3720331860 [2018-10-05 23:48:31.056734] I [MSGID: 122067] [ec-code.c:1046:ec_code_detect] 0-tier2-disperse-11: Using 'avx' CPU extensions [2018-10-05 23:48:31.058412] I [MSGID: 122067] [ec-code.c:1046:ec_code_detect] 0-tier2-disperse-10: Using 'avx' CPU extensions [2018-10-05 23:48:31.059866] I [MSGID: 122067] [ec-code.c:1046:ec_code_detect] 0-tier2-disperse-9: Using 'avx' CPU extensions [2018-10-05 23:48:31.061867] I [MSGID: 122067] [ec-code.c:1046:ec_code_detect] 0-tier2-disperse-8: Using 'avx' CPU extensions [2018-10-05 23:48:31.063638] I [MSGID: 122067] [ec-code.c:1046:ec_code_detect] 0-tier2-disperse-7: Using 'avx' CPU extensions [2018-10-05 23:48:31.065803] I [MSGID: 122067] [ec-code.c:1046:ec_code_detect] 0-tier2-disperse-6: Using 'avx' CPU extensions [2018-10-05 23:48:31.067600] I [MSGID: 122067] [ec-code.c:1046:ec_code_detect] 0-tier2-disperse-5: Using 'avx' CPU extensions [2018-10-05 23:48:31.069187] I [MSGID: 122067] [ec-code.c:1046:ec_code_detect] 0-tier2-disperse-4: Using 'avx' CPU extensions [2018-10-05 23:48:31.070800] I [MSGID: 122067] [ec-code.c:1046:ec_code_detect] 0-tier2-disperse-3: Using 'avx' CPU extensions [2018-10-05 23:48:31.072414] I [MSGID: 122067] [ec-code.c:1046:ec_code_detect] 0-tier2-disperse-2: Using 'avx' CPU extensions [2018-10-05 23:48:31.073942] I [MSGID: 122067] [ec-code.c:1046:ec_code_detect] 0-tier2-disperse-1: Using 'avx' CPU extensions [2018-10-05 23:48:31.075607] I [MSGID: 122067] [ec-code.c:1046:ec_code_detect] 0-tier2-disperse-0: Using 'avx' CPU extensions [2018-10-05 23:48:31.076549] I [MSGID: 101190] [event-epoll.c:613:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2 [2018-10-05 23:48:31.076579] I [MSGID: 101190] [event-epoll.c:613:event_dispatch_epoll_worker] 0-epoll: Started thread with index 3 [2018-10-05 23:48:31.076657] I [MSGID: 101190] [event-epoll.c:613:event_dispatch_epoll_worker] 0-epoll: Started thread with index 4 [2018-10-05 23:48:31.087379] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-0: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.089751] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-1: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.089992] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-0: changing port to 49152 (from 0) [2018-10-05 23:48:31.091771] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-2: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.094292] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:31.094984] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-0: Connected to tier2-client-0, attached to remote volume '/gluster/mnt1/brick'. [2018-10-05 23:48:31.095000] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-0: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:31.095122] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-0: Server lk version = 1 [2018-10-05 23:48:31.096769] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-3: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.098898] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-4: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.099061] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-3: changing port to 49153 (from 0) [2018-10-05 23:48:31.101017] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-5: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.103490] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-3: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:31.104123] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-3: Connected to tier2-client-3, attached to remote volume '/gluster/mnt2/brick'. [2018-10-05 23:48:31.104136] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-3: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:31.104239] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-3: Server lk version = 1 [2018-10-05 23:48:31.105628] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-6: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.107739] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-7: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.107882] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-6: changing port to 49154 (from 0) [2018-10-05 23:48:31.109811] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-8: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.112022] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-6: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:31.112588] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-6: Connected to tier2-client-6, attached to remote volume '/gluster/mnt3/brick'. [2018-10-05 23:48:31.112602] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-6: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:31.112669] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-6: Server lk version = 1 [2018-10-05 23:48:31.114119] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-9: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.116154] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-10: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.116285] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-9: changing port to 49155 (from 0) [2018-10-05 23:48:31.118610] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-11: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.121237] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-9: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:31.121822] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-9: Connected to tier2-client-9, attached to remote volume '/gluster/mnt4/brick'. [2018-10-05 23:48:31.121840] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-9: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:31.121907] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-9: Server lk version = 1 [2018-10-05 23:48:31.123413] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-12: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.125487] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-13: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.125647] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-12: changing port to 49156 (from 0) [2018-10-05 23:48:31.127773] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-14: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.129989] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-12: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:31.130474] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-12: Connected to tier2-client-12, attached to remote volume '/gluster/mnt5/brick'. [2018-10-05 23:48:31.130491] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-12: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:31.130538] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-12: Server lk version = 1 [2018-10-05 23:48:31.131975] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-15: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.134060] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-16: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.134188] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-15: changing port to 49157 (from 0) [2018-10-05 23:48:31.136310] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-17: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.138736] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-15: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:31.139275] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-15: Connected to tier2-client-15, attached to remote volume '/gluster/mnt6/brick'. [2018-10-05 23:48:31.139292] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-15: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:31.139364] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-15: Server lk version = 1 [2018-10-05 23:48:31.140967] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-18: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.143099] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-19: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.143203] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-18: changing port to 49158 (from 0) [2018-10-05 23:48:31.145140] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-20: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.147349] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-18: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:31.147807] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-18: Connected to tier2-client-18, attached to remote volume '/gluster/mnt7/brick'. [2018-10-05 23:48:31.147824] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-18: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:31.147901] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-18: Server lk version = 1 [2018-10-05 23:48:31.149267] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-21: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.151777] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-22: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.151916] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-21: changing port to 49159 (from 0) [2018-10-05 23:48:31.153850] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-23: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.156108] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-21: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:31.156610] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-21: Connected to tier2-client-21, attached to remote volume '/gluster/mnt8/brick'. [2018-10-05 23:48:31.156629] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-21: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:31.156696] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-21: Server lk version = 1 [2018-10-05 23:48:31.158038] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-24: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.160104] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-25: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.160225] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-24: changing port to 49160 (from 0) [2018-10-05 23:48:31.162236] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-26: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.164451] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-24: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:31.164880] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-24: Connected to tier2-client-24, attached to remote volume '/gluster/mnt9/brick'. [2018-10-05 23:48:31.164895] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-24: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:31.164967] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-24: Server lk version = 1 [2018-10-05 23:48:31.166801] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-27: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.168880] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-28: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.169027] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-27: changing port to 49161 (from 0) [2018-10-05 23:48:31.170952] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-29: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.173169] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-27: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:31.173669] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-27: Connected to tier2-client-27, attached to remote volume '/gluster/mnt10/brick'. [2018-10-05 23:48:31.173682] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-27: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:31.173761] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-27: Server lk version = 1 [2018-10-05 23:48:31.175161] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-30: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.177256] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-31: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.177368] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-30: changing port to 49162 (from 0) [2018-10-05 23:48:31.179725] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-32: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.181959] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-30: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:31.182499] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-30: Connected to tier2-client-30, attached to remote volume '/gluster/mnt11/brick'. [2018-10-05 23:48:31.182517] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-30: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:31.182586] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-30: Server lk version = 1 [2018-10-05 23:48:31.184108] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-33: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.186209] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-34: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.186316] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-33: changing port to 49163 (from 0) [2018-10-05 23:48:31.188429] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-35: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.190730] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-33: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:31.191132] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-33: Connected to tier2-client-33, attached to remote volume '/gluster/mnt12/brick'. [2018-10-05 23:48:31.191145] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-33: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:31.191211] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-33: Server lk version = 1 [2018-10-05 23:48:31.192751] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-36: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.194796] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-37: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.197083] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-38: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.199221] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-39: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.201266] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-40: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.203287] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-41: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.205370] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-42: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.207392] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-43: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.209302] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-44: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.211684] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-45: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.213765] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-46: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.215767] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-47: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.217717] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-48: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.219697] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-49: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.221635] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-50: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.223664] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-51: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.226075] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-52: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.228159] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-53: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.230211] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-54: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.232299] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-55: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.234470] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-56: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.236593] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-57: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.238773] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-58: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.240874] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-59: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.242922] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-60: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.244918] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-61: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.246890] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-62: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.248878] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-63: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.251234] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-64: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.253245] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-65: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.255308] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-66: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.257529] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-67: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.259576] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-68: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.261625] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-69: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.263676] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-70: parent translators are ready, attempting connect on transport [2018-10-05 23:48:31.265632] I [MSGID: 114020] [client.c:2360:notify] 0-tier2-client-71: parent translators are ready, attempting connect on transport Final graph: +------------------------------------------------------------------------------+ 1: volume tier2-client-0 2: type protocol/client 3: option opversion 31202 4: option clnt-lk-version 1 5: option volfile-checksum 0 6: option volfile-key rebalance/tier2 7: option client-version 3.12.14 8: option process-uuid s01-4642-2018/10/05-23:48:26:48007-tier2-client-0-0-0 9: option fops-version 1298437 10: option ping-timeout 42 11: option remote-host s01-stg 12: option remote-subvolume /gluster/mnt1/brick 13: option transport-type socket 14: option transport.address-family inet 15: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 16: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 17: option event-threads 4 18: option transport.tcp-user-timeout 0 19: option transport.socket.keepalive-time 20 20: option transport.socket.keepalive-interval 2 21: option transport.socket.keepalive-count 9 22: end-volume 23: 24: volume tier2-client-1 25: type protocol/client 26: option ping-timeout 42 27: option remote-host s02-stg 28: option remote-subvolume /gluster/mnt1/brick 29: option transport-type socket 30: option transport.address-family inet 31: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 32: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 33: option event-threads 4 34: option transport.tcp-user-timeout 0 35: option transport.socket.keepalive-time 20 36: option transport.socket.keepalive-interval 2 37: option transport.socket.keepalive-count 9 38: end-volume 39: 40: volume tier2-client-2 41: type protocol/client 42: option ping-timeout 42 43: option remote-host s03-stg 44: option remote-subvolume /gluster/mnt1/brick 45: option transport-type socket 46: option transport.address-family inet 47: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 48: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 49: option event-threads 4 50: option transport.tcp-user-timeout 0 51: option transport.socket.keepalive-time 20 52: option transport.socket.keepalive-interval 2 53: option transport.socket.keepalive-count 9 54: end-volume 55: 56: volume tier2-client-3 57: type protocol/client 58: option opversion 31202 59: option clnt-lk-version 1 60: option volfile-checksum 0 61: option volfile-key rebalance/tier2 62: option client-version 3.12.14 63: option process-uuid s01-4642-2018/10/05-23:48:26:48007-tier2-client-3-0-0 64: option fops-version 1298437 65: option ping-timeout 42 66: option remote-host s01-stg 67: option remote-subvolume /gluster/mnt2/brick 68: option transport-type socket 69: option transport.address-family inet 70: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 71: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 72: option event-threads 4 73: option transport.tcp-user-timeout 0 74: option transport.socket.keepalive-time 20 75: option transport.socket.keepalive-interval 2 76: option transport.socket.keepalive-count 9 77: end-volume 78: 79: volume tier2-client-4 80: type protocol/client 81: option ping-timeout 42 82: option remote-host s02-stg 83: option remote-subvolume /gluster/mnt2/brick 84: option transport-type socket 85: option transport.address-family inet 86: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 87: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 88: option event-threads 4 89: option transport.tcp-user-timeout 0 90: option transport.socket.keepalive-time 20 91: option transport.socket.keepalive-interval 2 92: option transport.socket.keepalive-count 9 93: end-volume 94: 95: volume tier2-client-5 96: type protocol/client 97: option ping-timeout 42 98: option remote-host s03-stg 99: option remote-subvolume /gluster/mnt2/brick 100: option transport-type socket 101: option transport.address-family inet 102: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 103: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 104: option event-threads 4 105: option transport.tcp-user-timeout 0 106: option transport.socket.keepalive-time 20 107: option transport.socket.keepalive-interval 2 108: option transport.socket.keepalive-count 9 109: end-volume 110: 111: volume tier2-disperse-0 112: type cluster/disperse 113: option redundancy 2 114: option cpu-extensions auto 115: subvolumes tier2-client-0 tier2-client-1 tier2-client-2 tier2-client-3 tier2-client-4 tier2-client-5 116: end-volume 117: 118: volume tier2-client-6 119: type protocol/client 120: option opversion 31202 121: option clnt-lk-version 1 122: option volfile-checksum 0 123: option volfile-key rebalance/tier2 124: option client-version 3.12.14 125: option process-uuid s01-4642-2018/10/05-23:48:26:48007-tier2-client-6-0-0 126: option fops-version 1298437 127: option ping-timeout 42 128: option remote-host s01-stg 129: option remote-subvolume /gluster/mnt3/brick 130: option transport-type socket 131: option transport.address-family inet 132: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 133: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 134: option event-threads 4 135: option transport.tcp-user-timeout 0 136: option transport.socket.keepalive-time 20 137: option transport.socket.keepalive-interval 2 138: option transport.socket.keepalive-count 9 139: end-volume 140: 141: volume tier2-client-7 142: type protocol/client 143: option ping-timeout 42 144: option remote-host s02-stg 145: option remote-subvolume /gluster/mnt3/brick 146: option transport-type socket 147: option transport.address-family inet 148: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 149: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 150: option event-threads 4 151: option transport.tcp-user-timeout 0 152: option transport.socket.keepalive-time 20 153: option transport.socket.keepalive-interval 2 154: option transport.socket.keepalive-count 9 155: end-volume 156: 157: volume tier2-client-8 158: type protocol/client 159: option ping-timeout 42 160: option remote-host s03-stg 161: option remote-subvolume /gluster/mnt3/brick 162: option transport-type socket 163: option transport.address-family inet 164: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 165: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 166: option event-threads 4 167: option transport.tcp-user-timeout 0 168: option transport.socket.keepalive-time 20 169: option transport.socket.keepalive-interval 2 170: option transport.socket.keepalive-count 9 171: end-volume 172: 173: volume tier2-client-9 174: type protocol/client 175: option opversion 31202 176: option clnt-lk-version 1 177: option volfile-checksum 0 178: option volfile-key rebalance/tier2 179: option client-version 3.12.14 180: option process-uuid s01-4642-2018/10/05-23:48:26:48007-tier2-client-9-0-0 181: option fops-version 1298437 182: option ping-timeout 42 183: option remote-host s01-stg 184: option remote-subvolume /gluster/mnt4/brick 185: option transport-type socket 186: option transport.address-family inet 187: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 188: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 189: option event-threads 4 190: option transport.tcp-user-timeout 0 191: option transport.socket.keepalive-time 20 192: option transport.socket.keepalive-interval 2 193: option transport.socket.keepalive-count 9 194: end-volume 195: 196: volume tier2-client-10 197: type protocol/client 198: option ping-timeout 42 199: option remote-host s02-stg 200: option remote-subvolume /gluster/mnt4/brick 201: option transport-type socket 202: option transport.address-family inet 203: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 204: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 205: option event-threads 4 206: option transport.tcp-user-timeout 0 207: option transport.socket.keepalive-time 20 208: option transport.socket.keepalive-interval 2 209: option transport.socket.keepalive-count 9 210: end-volume 211: 212: volume tier2-client-11 213: type protocol/client 214: option ping-timeout 42 215: option remote-host s03-stg 216: option remote-subvolume /gluster/mnt4/brick 217: option transport-type socket 218: option transport.address-family inet 219: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 220: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 221: option event-threads 4 222: option transport.tcp-user-timeout 0 223: option transport.socket.keepalive-time 20 224: option transport.socket.keepalive-interval 2 225: option transport.socket.keepalive-count 9 226: end-volume 227: 228: volume tier2-disperse-1 229: type cluster/disperse 230: option redundancy 2 231: option cpu-extensions auto 232: subvolumes tier2-client-6 tier2-client-7 tier2-client-8 tier2-client-9 tier2-client-10 tier2-client-11 233: end-volume 234: 235: volume tier2-client-12 236: type protocol/client 237: option opversion 31202 238: option clnt-lk-version 1 239: option volfile-checksum 0 240: option volfile-key rebalance/tier2 241: option client-version 3.12.14 242: option process-uuid s01-4642-2018/10/05-23:48:26:48007-tier2-client-12-0-0 243: option fops-version 1298437 244: option ping-timeout 42 245: option remote-host s01-stg 246: option remote-subvolume /gluster/mnt5/brick 247: option transport-type socket 248: option transport.address-family inet 249: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 250: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 251: option event-threads 4 252: option transport.tcp-user-timeout 0 253: option transport.socket.keepalive-time 20 254: option transport.socket.keepalive-interval 2 255: option transport.socket.keepalive-count 9 256: end-volume 257: 258: volume tier2-client-13 259: type protocol/client 260: option ping-timeout 42 261: option remote-host s02-stg 262: option remote-subvolume /gluster/mnt5/brick 263: option transport-type socket 264: option transport.address-family inet 265: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 266: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 267: option event-threads 4 268: option transport.tcp-user-timeout 0 269: option transport.socket.keepalive-time 20 270: option transport.socket.keepalive-interval 2 271: option transport.socket.keepalive-count 9 272: end-volume 273: 274: volume tier2-client-14 275: type protocol/client 276: option ping-timeout 42 277: option remote-host s03-stg 278: option remote-subvolume /gluster/mnt5/brick 279: option transport-type socket 280: option transport.address-family inet 281: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 282: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 283: option event-threads 4 284: option transport.tcp-user-timeout 0 285: option transport.socket.keepalive-time 20 286: option transport.socket.keepalive-interval 2 287: option transport.socket.keepalive-count 9 288: end-volume 289: 290: volume tier2-client-15 291: type protocol/client 292: option opversion 31202 293: option clnt-lk-version 1 294: option volfile-checksum 0 295: option volfile-key rebalance/tier2 296: option client-version 3.12.14 297: option process-uuid s01-4642-2018/10/05-23:48:26:48007-tier2-client-15-0-0 298: option fops-version 1298437 299: option ping-timeout 42 300: option remote-host s01-stg 301: option remote-subvolume /gluster/mnt6/brick 302: option transport-type socket 303: option transport.address-family inet 304: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 305: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 306: option event-threads 4 307: option transport.tcp-user-timeout 0 308: option transport.socket.keepalive-time 20 309: option transport.socket.keepalive-interval 2 310: option transport.socket.keepalive-count 9 311: end-volume 312: 313: volume tier2-client-16 314: type protocol/client 315: option ping-timeout 42 316: option remote-host s02-stg 317: option remote-subvolume /gluster/mnt6/brick 318: option transport-type socket 319: option transport.address-family inet 320: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 321: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 322: option event-threads 4 323: option transport.tcp-user-timeout 0 324: option transport.socket.keepalive-time 20 325: option transport.socket.keepalive-interval 2 326: option transport.socket.keepalive-count 9 327: end-volume 328: 329: volume tier2-client-17 330: type protocol/client 331: option ping-timeout 42 332: option remote-host s03-stg 333: option remote-subvolume /gluster/mnt6/brick 334: option transport-type socket 335: option transport.address-family inet 336: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 337: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 338: option event-threads 4 339: option transport.tcp-user-timeout 0 340: option transport.socket.keepalive-time 20 341: option transport.socket.keepalive-interval 2 342: option transport.socket.keepalive-count 9 343: end-volume 344: 345: volume tier2-disperse-2 346: type cluster/disperse 347: option redundancy 2 348: option cpu-extensions auto 349: subvolumes tier2-client-12 tier2-client-13 tier2-client-14 tier2-client-15 tier2-client-16 tier2-client-17 350: end-volume 351: 352: volume tier2-client-18 353: type protocol/client 354: option opversion 31202 355: option clnt-lk-version 1 356: option volfile-checksum 0 357: option volfile-key rebalance/tier2 358: option client-version 3.12.14 359: option process-uuid s01-4642-2018/10/05-23:48:26:48007-tier2-client-18-0-0 360: option fops-version 1298437 361: option ping-timeout 42 362: option remote-host s01-stg 363: option remote-subvolume /gluster/mnt7/brick 364: option transport-type socket 365: option transport.address-family inet 366: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 367: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 368: option event-threads 4 369: option transport.tcp-user-timeout 0 370: option transport.socket.keepalive-time 20 371: option transport.socket.keepalive-interval 2 372: option transport.socket.keepalive-count 9 373: end-volume 374: 375: volume tier2-client-19 376: type protocol/client 377: option ping-timeout 42 378: option remote-host s02-stg 379: option remote-subvolume /gluster/mnt7/brick 380: option transport-type socket 381: option transport.address-family inet 382: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 383: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 384: option event-threads 4 385: option transport.tcp-user-timeout 0 386: option transport.socket.keepalive-time 20 387: option transport.socket.keepalive-interval 2 388: option transport.socket.keepalive-count 9 389: end-volume 390: 391: volume tier2-client-20 392: type protocol/client 393: option ping-timeout 42 394: option remote-host s03-stg 395: option remote-subvolume /gluster/mnt7/brick 396: option transport-type socket 397: option transport.address-family inet 398: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 399: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 400: option event-threads 4 401: option transport.tcp-user-timeout 0 402: option transport.socket.keepalive-time 20 403: option transport.socket.keepalive-interval 2 404: option transport.socket.keepalive-count 9 405: end-volume 406: 407: volume tier2-client-21 408: type protocol/client 409: option opversion 31202 410: option clnt-lk-version 1 411: option volfile-checksum 0 412: option volfile-key rebalance/tier2 413: option client-version 3.12.14 414: option process-uuid s01-4642-2018/10/05-23:48:26:48007-tier2-client-21-0-0 415: option fops-version 1298437 416: option ping-timeout 42 417: option remote-host s01-stg 418: option remote-subvolume /gluster/mnt8/brick 419: option transport-type socket 420: option transport.address-family inet 421: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 422: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 423: option event-threads 4 424: option transport.tcp-user-timeout 0 425: option transport.socket.keepalive-time 20 426: option transport.socket.keepalive-interval 2 427: option transport.socket.keepalive-count 9 428: end-volume 429: 430: volume tier2-client-22 431: type protocol/client 432: option ping-timeout 42 433: option remote-host s02-stg 434: option remote-subvolume /gluster/mnt8/brick 435: option transport-type socket 436: option transport.address-family inet 437: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 438: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 439: option event-threads 4 440: option transport.tcp-user-timeout 0 441: option transport.socket.keepalive-time 20 442: option transport.socket.keepalive-interval 2 443: option transport.socket.keepalive-count 9 444: end-volume 445: 446: volume tier2-client-23 447: type protocol/client 448: option ping-timeout 42 449: option remote-host s03-stg 450: option remote-subvolume /gluster/mnt8/brick 451: option transport-type socket 452: option transport.address-family inet 453: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 454: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 455: option event-threads 4 456: option transport.tcp-user-timeout 0 457: option transport.socket.keepalive-time 20 458: option transport.socket.keepalive-interval 2 459: option transport.socket.keepalive-count 9 460: end-volume 461: 462: volume tier2-disperse-3 463: type cluster/disperse 464: option redundancy 2 465: option cpu-extensions auto 466: subvolumes tier2-client-18 tier2-client-19 tier2-client-20 tier2-client-21 tier2-client-22 tier2-client-23 467: end-volume 468: 469: volume tier2-client-24 470: type protocol/client 471: option opversion 31202 472: option clnt-lk-version 1 473: option volfile-checksum 0 474: option volfile-key rebalance/tier2 475: option client-version 3.12.14 476: option process-uuid s01-4642-2018/10/05-23:48:26:48007-tier2-client-24-0-0 477: option fops-version 1298437 478: option ping-timeout 42 479: option remote-host s01-stg 480: option remote-subvolume /gluster/mnt9/brick 481: option transport-type socket 482: option transport.address-family inet 483: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 484: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 485: option event-threads 4 486: option transport.tcp-user-timeout 0 487: option transport.socket.keepalive-time 20 488: option transport.socket.keepalive-interval 2 489: option transport.socket.keepalive-count 9 490: end-volume 491: 492: volume tier2-client-25 493: type protocol/client 494: option ping-timeout 42 495: option remote-host s02-stg 496: option remote-subvolume /gluster/mnt9/brick 497: option transport-type socket 498: option transport.address-family inet 499: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 500: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 501: option event-threads 4 502: option transport.tcp-user-timeout 0 503: option transport.socket.keepalive-time 20 504: option transport.socket.keepalive-interval 2 505: option transport.socket.keepalive-count 9 506: end-volume 507: 508: volume tier2-client-26 509: type protocol/client 510: option ping-timeout 42 511: option remote-host s03-stg 512: option remote-subvolume /gluster/mnt9/brick 513: option transport-type socket 514: option transport.address-family inet 515: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 516: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 517: option event-threads 4 518: option transport.tcp-user-timeout 0 519: option transport.socket.keepalive-time 20 520: option transport.socket.keepalive-interval 2 521: option transport.socket.keepalive-count 9 522: end-volume 523: 524: volume tier2-client-27 525: type protocol/client 526: option opversion 31202 527: option clnt-lk-version 1 528: option volfile-checksum 0 529: option volfile-key rebalance/tier2 530: option client-version 3.12.14 531: option process-uuid s01-4642-2018/10/05-23:48:26:48007-tier2-client-27-0-0 532: option fops-version 1298437 533: option ping-timeout 42 534: option remote-host s01-stg 535: option remote-subvolume /gluster/mnt10/brick 536: option transport-type socket 537: option transport.address-family inet 538: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 539: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 540: option event-threads 4 541: option transport.tcp-user-timeout 0 542: option transport.socket.keepalive-time 20 543: option transport.socket.keepalive-interval 2 544: option transport.socket.keepalive-count 9 545: end-volume 546: 547: volume tier2-client-28 548: type protocol/client 549: option ping-timeout 42 550: option remote-host s02-stg 551: option remote-subvolume /gluster/mnt10/brick 552: option transport-type socket 553: option transport.address-family inet 554: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 555: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 556: option event-threads 4 557: option transport.tcp-user-timeout 0 558: option transport.socket.keepalive-time 20 559: option transport.socket.keepalive-interval 2 560: option transport.socket.keepalive-count 9 561: end-volume 562: 563: volume tier2-client-29 564: type protocol/client 565: option ping-timeout 42 566: option remote-host s03-stg 567: option remote-subvolume /gluster/mnt10/brick 568: option transport-type socket 569: option transport.address-family inet 570: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 571: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 572: option event-threads 4 573: option transport.tcp-user-timeout 0 574: option transport.socket.keepalive-time 20 575: option transport.socket.keepalive-interval 2 576: option transport.socket.keepalive-count 9 577: end-volume 578: 579: volume tier2-disperse-4 580: type cluster/disperse 581: option redundancy 2 582: option cpu-extensions auto 583: subvolumes tier2-client-24 tier2-client-25 tier2-client-26 tier2-client-27 tier2-client-28 tier2-client-29 584: end-volume 585: 586: volume tier2-client-30 587: type protocol/client 588: option opversion 31202 589: option clnt-lk-version 1 590: option volfile-checksum 0 591: option volfile-key rebalance/tier2 592: option client-version 3.12.14 593: option process-uuid s01-4642-2018/10/05-23:48:26:48007-tier2-client-30-0-0 594: option fops-version 1298437 595: option ping-timeout 42 596: option remote-host s01-stg 597: option remote-subvolume /gluster/mnt11/brick 598: option transport-type socket 599: option transport.address-family inet 600: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 601: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 602: option event-threads 4 603: option transport.tcp-user-timeout 0 604: option transport.socket.keepalive-time 20 605: option transport.socket.keepalive-interval 2 606: option transport.socket.keepalive-count 9 607: end-volume 608: 609: volume tier2-client-31 610: type protocol/client 611: option ping-timeout 42 612: option remote-host s02-stg 613: option remote-subvolume /gluster/mnt11/brick 614: option transport-type socket 615: option transport.address-family inet 616: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 617: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 618: option event-threads 4 619: option transport.tcp-user-timeout 0 620: option transport.socket.keepalive-time 20 621: option transport.socket.keepalive-interval 2 622: option transport.socket.keepalive-count 9 623: end-volume 624: 625: volume tier2-client-32 626: type protocol/client 627: option ping-timeout 42 628: option remote-host s03-stg 629: option remote-subvolume /gluster/mnt11/brick 630: option transport-type socket 631: option transport.address-family inet 632: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 633: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 634: option event-threads 4 635: option transport.tcp-user-timeout 0 636: option transport.socket.keepalive-time 20 637: option transport.socket.keepalive-interval 2 638: option transport.socket.keepalive-count 9 639: end-volume 640: 641: volume tier2-client-33 642: type protocol/client 643: option opversion 31202 644: option clnt-lk-version 1 645: option volfile-checksum 0 646: option volfile-key rebalance/tier2 647: option client-version 3.12.14 648: option process-uuid s01-4642-2018/10/05-23:48:26:48007-tier2-client-33-0-0 649: option fops-version 1298437 650: option ping-timeout 42 651: option remote-host s01-stg 652: option remote-subvolume /gluster/mnt12/brick 653: option transport-type socket 654: option transport.address-family inet 655: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 656: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 657: option event-threads 4 658: option transport.tcp-user-timeout 0 659: option transport.socket.keepalive-time 20 660: option transport.socket.keepalive-interval 2 661: option transport.socket.keepalive-count 9 662: end-volume 663: 664: volume tier2-client-34 665: type protocol/client 666: option ping-timeout 42 667: option remote-host s02-stg 668: option remote-subvolume /gluster/mnt12/brick 669: option transport-type socket 670: option transport.address-family inet 671: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 672: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 673: option event-threads 4 674: option transport.tcp-user-timeout 0 675: option transport.socket.keepalive-time 20 676: option transport.socket.keepalive-interval 2 677: option transport.socket.keepalive-count 9 678: end-volume 679: 680: volume tier2-client-35 681: type protocol/client 682: option ping-timeout 42 683: option remote-host s03-stg 684: option remote-subvolume /gluster/mnt12/brick 685: option transport-type socket 686: option transport.address-family inet 687: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 688: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 689: option event-threads 4 690: option transport.tcp-user-timeout 0 691: option transport.socket.keepalive-time 20 692: option transport.socket.keepalive-interval 2 693: option transport.socket.keepalive-count 9 694: end-volume 695: 696: volume tier2-disperse-5 697: type cluster/disperse 698: option redundancy 2 699: option cpu-extensions auto 700: subvolumes tier2-client-30 tier2-client-31 tier2-client-32 tier2-client-33 tier2-client-34 tier2-client-35 701: end-volume 702: 703: volume tier2-client-36 704: type protocol/client 705: option ping-timeout 42 706: option remote-host s04-stg 707: option remote-subvolume /gluster/mnt1/brick 708: option transport-type socket 709: option transport.address-family inet 710: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 711: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 712: option event-threads 4 713: option transport.tcp-user-timeout 0 714: option transport.socket.keepalive-time 20 715: option transport.socket.keepalive-interval 2 716: option transport.socket.keepalive-count 9 717: end-volume 718: 719: volume tier2-client-37 720: type protocol/client 721: option ping-timeout 42 722: option remote-host s05-stg 723: option remote-subvolume /gluster/mnt1/brick 724: option transport-type socket 725: option transport.address-family inet 726: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 727: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 728: option event-threads 4 729: option transport.tcp-user-timeout 0 730: option transport.socket.keepalive-time 20 731: option transport.socket.keepalive-interval 2 732: option transport.socket.keepalive-count 9 733: end-volume 734: 735: volume tier2-client-38 736: type protocol/client 737: option ping-timeout 42 738: option remote-host s06-stg 739: option remote-subvolume /gluster/mnt1/brick 740: option transport-type socket 741: option transport.address-family inet 742: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 743: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 744: option event-threads 4 745: option transport.tcp-user-timeout 0 746: option transport.socket.keepalive-time 20 747: option transport.socket.keepalive-interval 2 748: option transport.socket.keepalive-count 9 749: end-volume 750: 751: volume tier2-client-39 752: type protocol/client 753: option ping-timeout 42 754: option remote-host s04-stg 755: option remote-subvolume /gluster/mnt2/brick 756: option transport-type socket 757: option transport.address-family inet 758: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 759: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 760: option event-threads 4 761: option transport.tcp-user-timeout 0 762: option transport.socket.keepalive-time 20 763: option transport.socket.keepalive-interval 2 764: option transport.socket.keepalive-count 9 765: end-volume 766: 767: volume tier2-client-40 768: type protocol/client 769: option ping-timeout 42 770: option remote-host s05-stg 771: option remote-subvolume /gluster/mnt2/brick 772: option transport-type socket 773: option transport.address-family inet 774: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 775: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 776: option event-threads 4 777: option transport.tcp-user-timeout 0 778: option transport.socket.keepalive-time 20 779: option transport.socket.keepalive-interval 2 780: option transport.socket.keepalive-count 9 781: end-volume 782: 783: volume tier2-client-41 784: type protocol/client 785: option ping-timeout 42 786: option remote-host s06-stg 787: option remote-subvolume /gluster/mnt2/brick 788: option transport-type socket 789: option transport.address-family inet 790: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 791: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 792: option event-threads 4 793: option transport.tcp-user-timeout 0 794: option transport.socket.keepalive-time 20 795: option transport.socket.keepalive-interval 2 796: option transport.socket.keepalive-count 9 797: end-volume 798: 799: volume tier2-disperse-6 800: type cluster/disperse 801: option redundancy 2 802: option cpu-extensions auto 803: subvolumes tier2-client-36 tier2-client-37 tier2-client-38 tier2-client-39 tier2-client-40 tier2-client-41 804: end-volume 805: 806: volume tier2-client-42 807: type protocol/client 808: option ping-timeout 42 809: option remote-host s04-stg 810: option remote-subvolume /gluster/mnt3/brick 811: option transport-type socket 812: option transport.address-family inet 813: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 814: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 815: option event-threads 4 816: option transport.tcp-user-timeout 0 817: option transport.socket.keepalive-time 20 818: option transport.socket.keepalive-interval 2 819: option transport.socket.keepalive-count 9 820: end-volume 821: 822: volume tier2-client-43 823: type protocol/client 824: option ping-timeout 42 825: option remote-host s05-stg 826: option remote-subvolume /gluster/mnt3/brick 827: option transport-type socket 828: option transport.address-family inet 829: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 830: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 831: option event-threads 4 832: option transport.tcp-user-timeout 0 833: option transport.socket.keepalive-time 20 834: option transport.socket.keepalive-interval 2 835: option transport.socket.keepalive-count 9 836: end-volume 837: 838: volume tier2-client-44 839: type protocol/client 840: option ping-timeout 42 841: option remote-host s06-stg 842: option remote-subvolume /gluster/mnt3/brick 843: option transport-type socket 844: option transport.address-family inet 845: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 846: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 847: option event-threads 4 848: option transport.tcp-user-timeout 0 849: option transport.socket.keepalive-time 20 850: option transport.socket.keepalive-interval 2 851: option transport.socket.keepalive-count 9 852: end-volume 853: 854: volume tier2-client-45 855: type protocol/client 856: option ping-timeout 42 857: option remote-host s04-stg 858: option remote-subvolume /gluster/mnt4/brick 859: option transport-type socket 860: option transport.address-family inet 861: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 862: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 863: option event-threads 4 864: option transport.tcp-user-timeout 0 865: option transport.socket.keepalive-time 20 866: option transport.socket.keepalive-interval 2 867: option transport.socket.keepalive-count 9 868: end-volume 869: 870: volume tier2-client-46 871: type protocol/client 872: option ping-timeout 42 873: option remote-host s05-stg 874: option remote-subvolume /gluster/mnt4/brick 875: option transport-type socket 876: option transport.address-family inet 877: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 878: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 879: option event-threads 4 880: option transport.tcp-user-timeout 0 881: option transport.socket.keepalive-time 20 882: option transport.socket.keepalive-interval 2 883: option transport.socket.keepalive-count 9 884: end-volume 885: 886: volume tier2-client-47 887: type protocol/client 888: option ping-timeout 42 889: option remote-host s06-stg 890: option remote-subvolume /gluster/mnt4/brick 891: option transport-type socket 892: option transport.address-family inet 893: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 894: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 895: option event-threads 4 896: option transport.tcp-user-timeout 0 897: option transport.socket.keepalive-time 20 898: option transport.socket.keepalive-interval 2 899: option transport.socket.keepalive-count 9 900: end-volume 901: 902: volume tier2-disperse-7 903: type cluster/disperse 904: option redundancy 2 905: option cpu-extensions auto 906: subvolumes tier2-client-42 tier2-client-43 tier2-client-44 tier2-client-45 tier2-client-46 tier2-client-47 907: end-volume 908: 909: volume tier2-client-48 910: type protocol/client 911: option ping-timeout 42 912: option remote-host s04-stg 913: option remote-subvolume /gluster/mnt5/brick 914: option transport-type socket 915: option transport.address-family inet 916: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 917: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 918: option event-threads 4 919: option transport.tcp-user-timeout 0 920: option transport.socket.keepalive-time 20 921: option transport.socket.keepalive-interval 2 922: option transport.socket.keepalive-count 9 923: end-volume 924: 925: volume tier2-client-49 926: type protocol/client 927: option ping-timeout 42 928: option remote-host s05-stg 929: option remote-subvolume /gluster/mnt5/brick 930: option transport-type socket 931: option transport.address-family inet 932: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 933: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 934: option event-threads 4 935: option transport.tcp-user-timeout 0 936: option transport.socket.keepalive-time 20 937: option transport.socket.keepalive-interval 2 938: option transport.socket.keepalive-count 9 939: end-volume 940: 941: volume tier2-client-50 942: type protocol/client 943: option ping-timeout 42 944: option remote-host s06-stg 945: option remote-subvolume /gluster/mnt5/brick 946: option transport-type socket 947: option transport.address-family inet 948: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 949: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 950: option event-threads 4 951: option transport.tcp-user-timeout 0 952: option transport.socket.keepalive-time 20 953: option transport.socket.keepalive-interval 2 954: option transport.socket.keepalive-count 9 955: end-volume 956: 957: volume tier2-client-51 958: type protocol/client 959: option ping-timeout 42 960: option remote-host s04-stg 961: option remote-subvolume /gluster/mnt6/brick 962: option transport-type socket 963: option transport.address-family inet 964: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 965: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 966: option event-threads 4 967: option transport.tcp-user-timeout 0 968: option transport.socket.keepalive-time 20 969: option transport.socket.keepalive-interval 2 970: option transport.socket.keepalive-count 9 971: end-volume 972: 973: volume tier2-client-52 974: type protocol/client 975: option ping-timeout 42 976: option remote-host s05-stg 977: option remote-subvolume /gluster/mnt6/brick 978: option transport-type socket 979: option transport.address-family inet 980: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 981: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 982: option event-threads 4 983: option transport.tcp-user-timeout 0 984: option transport.socket.keepalive-time 20 985: option transport.socket.keepalive-interval 2 986: option transport.socket.keepalive-count 9 987: end-volume 988: 989: volume tier2-client-53 990: type protocol/client 991: option ping-timeout 42 992: option remote-host s06-stg 993: option remote-subvolume /gluster/mnt6/brick 994: option transport-type socket 995: option transport.address-family inet 996: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 997: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 998: option event-threads 4 999: option transport.tcp-user-timeout 0 1000: option transport.socket.keepalive-time 20 1001: option transport.socket.keepalive-interval 2 1002: option transport.socket.keepalive-count 9 1003: end-volume 1004: 1005: volume tier2-disperse-8 1006: type cluster/disperse 1007: option redundancy 2 1008: option cpu-extensions auto 1009: subvolumes tier2-client-48 tier2-client-49 tier2-client-50 tier2-client-51 tier2-client-52 tier2-client-53 1010: end-volume 1011: 1012: volume tier2-client-54 1013: type protocol/client 1014: option ping-timeout 42 1015: option remote-host s04-stg 1016: option remote-subvolume /gluster/mnt7/brick 1017: option transport-type socket 1018: option transport.address-family inet 1019: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 1020: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 1021: option event-threads 4 1022: option transport.tcp-user-timeout 0 1023: option transport.socket.keepalive-time 20 1024: option transport.socket.keepalive-interval 2 1025: option transport.socket.keepalive-count 9 1026: end-volume 1027: 1028: volume tier2-client-55 1029: type protocol/client 1030: option ping-timeout 42 1031: option remote-host s05-stg 1032: option remote-subvolume /gluster/mnt7/brick 1033: option transport-type socket 1034: option transport.address-family inet 1035: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 1036: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 1037: option event-threads 4 1038: option transport.tcp-user-timeout 0 1039: option transport.socket.keepalive-time 20 1040: option transport.socket.keepalive-interval 2 1041: option transport.socket.keepalive-count 9 1042: end-volume 1043: 1044: volume tier2-client-56 1045: type protocol/client 1046: option ping-timeout 42 1047: option remote-host s06-stg 1048: option remote-subvolume /gluster/mnt7/brick 1049: option transport-type socket 1050: option transport.address-family inet 1051: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 1052: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 1053: option event-threads 4 1054: option transport.tcp-user-timeout 0 1055: option transport.socket.keepalive-time 20 1056: option transport.socket.keepalive-interval 2 1057: option transport.socket.keepalive-count 9 1058: end-volume 1059: 1060: volume tier2-client-57 1061: type protocol/client 1062: option ping-timeout 42 1063: option remote-host s04-stg 1064: option remote-subvolume /gluster/mnt8/brick 1065: option transport-type socket 1066: option transport.address-family inet 1067: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 1068: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 1069: option event-threads 4 1070: option transport.tcp-user-timeout 0 1071: option transport.socket.keepalive-time 20 1072: option transport.socket.keepalive-interval 2 1073: option transport.socket.keepalive-count 9 1074: end-volume 1075: 1076: volume tier2-client-58 1077: type protocol/client 1078: option ping-timeout 42 1079: option remote-host s05-stg 1080: option remote-subvolume /gluster/mnt8/brick 1081: option transport-type socket 1082: option transport.address-family inet 1083: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 1084: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 1085: option event-threads 4 1086: option transport.tcp-user-timeout 0 1087: option transport.socket.keepalive-time 20 1088: option transport.socket.keepalive-interval 2 1089: option transport.socket.keepalive-count 9 1090: end-volume 1091: 1092: volume tier2-client-59 1093: type protocol/client 1094: option ping-timeout 42 1095: option remote-host s06-stg 1096: option remote-subvolume /gluster/mnt8/brick 1097: option transport-type socket 1098: option transport.address-family inet 1099: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 1100: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 1101: option event-threads 4 1102: option transport.tcp-user-timeout 0 1103: option transport.socket.keepalive-time 20 1104: option transport.socket.keepalive-interval 2 1105: option transport.socket.keepalive-count 9 1106: end-volume 1107: 1108: volume tier2-disperse-9 1109: type cluster/disperse 1110: option redundancy 2 1111: option cpu-extensions auto 1112: subvolumes tier2-client-54 tier2-client-55 tier2-client-56 tier2-client-57 tier2-client-58 tier2-client-59 1113: end-volume 1114: 1115: volume tier2-client-60 1116: type protocol/client 1117: option ping-timeout 42 1118: option remote-host s04-stg 1119: option remote-subvolume /gluster/mnt9/brick 1120: option transport-type socket 1121: option transport.address-family inet 1122: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 1123: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 1124: option event-threads 4 1125: option transport.tcp-user-timeout 0 1126: option transport.socket.keepalive-time 20 1127: option transport.socket.keepalive-interval 2 1128: option transport.socket.keepalive-count 9 1129: end-volume 1130: 1131: volume tier2-client-61 1132: type protocol/client 1133: option ping-timeout 42 1134: option remote-host s05-stg 1135: option remote-subvolume /gluster/mnt9/brick 1136: option transport-type socket 1137: option transport.address-family inet 1138: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 1139: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 1140: option event-threads 4 1141: option transport.tcp-user-timeout 0 1142: option transport.socket.keepalive-time 20 1143: option transport.socket.keepalive-interval 2 1144: option transport.socket.keepalive-count 9 1145: end-volume 1146: 1147: volume tier2-client-62 1148: type protocol/client 1149: option ping-timeout 42 1150: option remote-host s06-stg 1151: option remote-subvolume /gluster/mnt9/brick 1152: option transport-type socket 1153: option transport.address-family inet 1154: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 1155: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 1156: option event-threads 4 1157: option transport.tcp-user-timeout 0 1158: option transport.socket.keepalive-time 20 1159: option transport.socket.keepalive-interval 2 1160: option transport.socket.keepalive-count 9 1161: end-volume 1162: 1163: volume tier2-client-63 1164: type protocol/client 1165: option ping-timeout 42 1166: option remote-host s04-stg 1167: option remote-subvolume /gluster/mnt10/brick 1168: option transport-type socket 1169: option transport.address-family inet 1170: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 1171: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 1172: option event-threads 4 1173: option transport.tcp-user-timeout 0 1174: option transport.socket.keepalive-time 20 1175: option transport.socket.keepalive-interval 2 1176: option transport.socket.keepalive-count 9 1177: end-volume 1178: 1179: volume tier2-client-64 1180: type protocol/client 1181: option ping-timeout 42 1182: option remote-host s05-stg 1183: option remote-subvolume /gluster/mnt10/brick 1184: option transport-type socket 1185: option transport.address-family inet 1186: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 1187: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 1188: option event-threads 4 1189: option transport.tcp-user-timeout 0 1190: option transport.socket.keepalive-time 20 1191: option transport.socket.keepalive-interval 2 1192: option transport.socket.keepalive-count 9 1193: end-volume 1194: 1195: volume tier2-client-65 1196: type protocol/client 1197: option ping-timeout 42 1198: option remote-host s06-stg 1199: option remote-subvolume /gluster/mnt10/brick 1200: option transport-type socket 1201: option transport.address-family inet 1202: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 1203: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 1204: option event-threads 4 1205: option transport.tcp-user-timeout 0 1206: option transport.socket.keepalive-time 20 1207: option transport.socket.keepalive-interval 2 1208: option transport.socket.keepalive-count 9 1209: end-volume 1210: 1211: volume tier2-disperse-10 1212: type cluster/disperse 1213: option redundancy 2 1214: option cpu-extensions auto 1215: subvolumes tier2-client-60 tier2-client-61 tier2-client-62 tier2-client-63 tier2-client-64 tier2-client-65 1216: end-volume 1217: 1218: volume tier2-client-66 1219: type protocol/client 1220: option ping-timeout 42 1221: option remote-host s04-stg 1222: option remote-subvolume /gluster/mnt11/brick 1223: option transport-type socket 1224: option transport.address-family inet 1225: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 1226: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 1227: option event-threads 4 1228: option transport.tcp-user-timeout 0 1229: option transport.socket.keepalive-time 20 1230: option transport.socket.keepalive-interval 2 1231: option transport.socket.keepalive-count 9 1232: end-volume 1233: 1234: volume tier2-client-67 1235: type protocol/client 1236: option ping-timeout 42 1237: option remote-host s05-stg 1238: option remote-subvolume /gluster/mnt11/brick 1239: option transport-type socket 1240: option transport.address-family inet 1241: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 1242: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 1243: option event-threads 4 1244: option transport.tcp-user-timeout 0 1245: option transport.socket.keepalive-time 20 1246: option transport.socket.keepalive-interval 2 1247: option transport.socket.keepalive-count 9 1248: end-volume 1249: 1250: volume tier2-client-68 1251: type protocol/client 1252: option ping-timeout 42 1253: option remote-host s06-stg 1254: option remote-subvolume /gluster/mnt11/brick 1255: option transport-type socket 1256: option transport.address-family inet 1257: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 1258: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 1259: option event-threads 4 1260: option transport.tcp-user-timeout 0 1261: option transport.socket.keepalive-time 20 1262: option transport.socket.keepalive-interval 2 1263: option transport.socket.keepalive-count 9 1264: end-volume 1265: 1266: volume tier2-client-69 1267: type protocol/client 1268: option ping-timeout 42 1269: option remote-host s04-stg 1270: option remote-subvolume /gluster/mnt12/brick 1271: option transport-type socket 1272: option transport.address-family inet 1273: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 1274: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 1275: option event-threads 4 1276: option transport.tcp-user-timeout 0 1277: option transport.socket.keepalive-time 20 1278: option transport.socket.keepalive-interval 2 1279: option transport.socket.keepalive-count 9 1280: end-volume 1281: 1282: volume tier2-client-70 1283: type protocol/client 1284: option ping-timeout 42 1285: option remote-host s05-stg 1286: option remote-subvolume /gluster/mnt12/brick 1287: option transport-type socket 1288: option transport.address-family inet 1289: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 1290: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 1291: option event-threads 4 1292: option transport.tcp-user-timeout 0 1293: option transport.socket.keepalive-time 20 1294: option transport.socket.keepalive-interval 2 1295: option transport.socket.keepalive-count 9 1296: end-volume 1297: 1298: volume tier2-client-71 1299: type protocol/client 1300: option ping-timeout 42 1301: option remote-host s06-stg 1302: option remote-subvolume /gluster/mnt12/brick 1303: option transport-type socket 1304: option transport.address-family inet 1305: option username a216f9c3-5819-402e-bdd8-d8ab25b6d404 1306: option password a7ec6bbd-a6e3-4b00-9d37-363bafc6df7a 1307: option event-threads 4 1308: option transport.tcp-user-timeout 0 1309: option transport.socket.keepalive-time 20 1310: option transport.socket.keepalive-interval 2 1311: option transport.socket.keepalive-count 9 1312: end-volume 1313: 1314: volume tier2-disperse-11 1315: type cluster/disperse 1316: option redundancy 2 1317: option cpu-extensions auto 1318: subvolumes tier2-client-66 tier2-client-67 tier2-client-68 tier2-client-69 tier2-client-70 tier2-client-71 1319: end-volume 1320: 1321: volume tier2-dht 1322: type cluster/distribute 1323: option use-readdirp yes 1324: option lookup-unhashed yes 1325: option assert-no-child-down yes 1326: option rebalance-cmd 1 1327: option node-uuid b734b083-4630-4523-9402-05d03565efee 1328: option commit-hash 3720331860 1329: option lookup-optimize on 1330: option min-free-disk 10 1331: option readdir-optimize on 1332: option lock-migration off 1333: subvolumes tier2-disperse-0 tier2-disperse-1 tier2-disperse-2 tier2-disperse-3 tier2-disperse-4 tier2-disperse-5 tier2-disperse-6 tier2-disperse-7 tier2-disperse-8 tier2-disperse-9 tier2-disperse-10 tier2-disperse-11 1334: end-volume 1335: 1336: volume tier2 1337: type debug/io-stats 1338: option log-level INFO 1339: option latency-measurement on 1340: option count-fop-hits on 1341: subvolumes tier2-dht 1342: end-volume 1343: +------------------------------------------------------------------------------+ [2018-10-05 23:48:36.055261] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-4: changing port to 49153 (from 0) [2018-10-05 23:48:36.055274] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-1: changing port to 49152 (from 0) [2018-10-05 23:48:36.055341] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-7: changing port to 49154 (from 0) [2018-10-05 23:48:36.055350] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-10: changing port to 49155 (from 0) [2018-10-05 23:48:36.055375] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-13: changing port to 49156 (from 0) [2018-10-05 23:48:36.060572] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-16: changing port to 49157 (from 0) [2018-10-05 23:48:36.060643] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-19: changing port to 49158 (from 0) [2018-10-05 23:48:36.065212] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-22: changing port to 49159 (from 0) [2018-10-05 23:48:36.065258] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-25: changing port to 49160 (from 0) [2018-10-05 23:48:36.065279] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-28: changing port to 49161 (from 0) [2018-10-05 23:48:36.065296] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-31: changing port to 49162 (from 0) [2018-10-05 23:48:36.065313] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-2: changing port to 49152 (from 0) [2018-10-05 23:48:36.065415] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-8: changing port to 49154 (from 0) [2018-10-05 23:48:36.065431] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-5: changing port to 49153 (from 0) [2018-10-05 23:48:36.065448] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-17: changing port to 49157 (from 0) [2018-10-05 23:48:36.065462] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-11: changing port to 49155 (from 0) [2018-10-05 23:48:36.065476] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-14: changing port to 49156 (from 0) [2018-10-05 23:48:36.065505] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-20: changing port to 49158 (from 0) [2018-10-05 23:48:36.065528] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-23: changing port to 49159 (from 0) [2018-10-05 23:48:36.065544] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-36: changing port to 49152 (from 0) [2018-10-05 23:48:36.065602] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-39: changing port to 49153 (from 0) [2018-10-05 23:48:36.065618] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-26: changing port to 49160 (from 0) [2018-10-05 23:48:36.065633] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-42: changing port to 49154 (from 0) [2018-10-05 23:48:36.065648] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-29: changing port to 49161 (from 0) [2018-10-05 23:48:36.065693] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-32: changing port to 49162 (from 0) [2018-10-05 23:48:36.065723] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-45: changing port to 49155 (from 0) [2018-10-05 23:48:36.065748] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-48: changing port to 49156 (from 0) [2018-10-05 23:48:36.065778] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-37: changing port to 49152 (from 0) [2018-10-05 23:48:36.065791] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-54: changing port to 49158 (from 0) [2018-10-05 23:48:36.065804] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-51: changing port to 49157 (from 0) [2018-10-05 23:48:36.065818] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-57: changing port to 49159 (from 0) [2018-10-05 23:48:36.065831] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-40: changing port to 49153 (from 0) [2018-10-05 23:48:36.065845] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-60: changing port to 49160 (from 0) [2018-10-05 23:48:36.065897] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-43: changing port to 49154 (from 0) [2018-10-05 23:48:36.065912] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-46: changing port to 49155 (from 0) [2018-10-05 23:48:36.065932] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-49: changing port to 49156 (from 0) [2018-10-05 23:48:36.065947] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-63: changing port to 49161 (from 0) [2018-10-05 23:48:36.065967] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-52: changing port to 49157 (from 0) [2018-10-05 23:48:36.065982] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-66: changing port to 49162 (from 0) [2018-10-05 23:48:36.065996] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-55: changing port to 49158 (from 0) [2018-10-05 23:48:36.066016] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-58: changing port to 49159 (from 0) [2018-10-05 23:48:36.066060] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-61: changing port to 49160 (from 0) [2018-10-05 23:48:36.066095] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-64: changing port to 49161 (from 0) [2018-10-05 23:48:36.066109] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-67: changing port to 49162 (from 0) [2018-10-05 23:48:36.066124] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-38: changing port to 49152 (from 0) [2018-10-05 23:48:36.066143] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-4: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.066204] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-41: changing port to 49153 (from 0) [2018-10-05 23:48:36.066231] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-47: changing port to 49155 (from 0) [2018-10-05 23:48:36.066246] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-44: changing port to 49154 (from 0) [2018-10-05 23:48:36.066260] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-50: changing port to 49156 (from 0) [2018-10-05 23:48:36.066275] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-53: changing port to 49157 (from 0) [2018-10-05 23:48:36.066289] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-56: changing port to 49158 (from 0) [2018-10-05 23:48:36.066316] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-59: changing port to 49159 (from 0) [2018-10-05 23:48:36.066388] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-62: changing port to 49160 (from 0) [2018-10-05 23:48:36.066404] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-65: changing port to 49161 (from 0) [2018-10-05 23:48:36.066418] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-68: changing port to 49162 (from 0) [2018-10-05 23:48:36.066458] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-7: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.183367] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-1: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.183470] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-4: Connected to tier2-client-4, attached to remote volume '/gluster/mnt2/brick'. [2018-10-05 23:48:36.183486] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-4: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.183573] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-7: Connected to tier2-client-7, attached to remote volume '/gluster/mnt3/brick'. [2018-10-05 23:48:36.183591] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-7: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.183665] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-10: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.184443] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-7: Server lk version = 1 [2018-10-05 23:48:36.184466] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-13: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.184521] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-16: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.184567] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-19: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.184609] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-22: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.184652] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-28: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.184689] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-25: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.184730] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-31: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.184770] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.184812] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-5: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.184852] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-17: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.184888] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-8: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.184930] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-11: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.184982] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-20: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.185035] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-1: Connected to tier2-client-1, attached to remote volume '/gluster/mnt1/brick'. [2018-10-05 23:48:36.185047] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-1: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.185105] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-14: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.185145] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-23: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.185187] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-36: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.185225] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-26: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.185268] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-29: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.185311] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-39: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.185353] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-32: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.185394] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-45: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.185431] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-42: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.185482] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-10: Connected to tier2-client-10, attached to remote volume '/gluster/mnt4/brick'. [2018-10-05 23:48:36.185493] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-10: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.185534] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-37: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.185572] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-54: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.185614] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-51: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.185656] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-48: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.185692] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-57: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.185732] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-43: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.185773] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-40: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.185816] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-60: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.185856] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-46: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.185902] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-49: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.185940] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-52: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.185985] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-63: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.186028] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-55: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.186080] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-61: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.186123] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-66: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.186141] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-67: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.186171] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-64: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.186219] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-58: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.186227] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-38: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.186285] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-47: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.186285] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-41: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.186333] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-4: Server lk version = 1 [2018-10-05 23:48:36.186357] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-44: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.186369] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-50: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.186413] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-53: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.186438] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-59: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.186466] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-56: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.186504] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-13: Connected to tier2-client-13, attached to remote volume '/gluster/mnt5/brick'. [2018-10-05 23:48:36.186519] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-13: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.186536] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-16: Connected to tier2-client-16, attached to remote volume '/gluster/mnt6/brick'. [2018-10-05 23:48:36.186550] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-16: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.186574] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-19: Connected to tier2-client-19, attached to remote volume '/gluster/mnt7/brick'. [2018-10-05 23:48:36.186589] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-19: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.186608] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-22: Connected to tier2-client-22, attached to remote volume '/gluster/mnt8/brick'. [2018-10-05 23:48:36.186622] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-22: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.186640] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-28: Connected to tier2-client-28, attached to remote volume '/gluster/mnt10/brick'. [2018-10-05 23:48:36.186655] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-28: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.186669] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-1: Server lk version = 1 [2018-10-05 23:48:36.186697] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-25: Connected to tier2-client-25, attached to remote volume '/gluster/mnt9/brick'. [2018-10-05 23:48:36.186708] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-25: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.186712] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-5: Connected to tier2-client-5, attached to remote volume '/gluster/mnt2/brick'. [2018-10-05 23:48:36.186729] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-5: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.186755] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-2: Connected to tier2-client-2, attached to remote volume '/gluster/mnt1/brick'. [2018-10-05 23:48:36.186768] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-2: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.186779] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-31: Connected to tier2-client-31, attached to remote volume '/gluster/mnt11/brick'. [2018-10-05 23:48:36.186794] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-31: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.186799] I [MSGID: 122061] [ec.c:343:ec_up] 0-tier2-disperse-0: Going UP [2018-10-05 23:48:36.187094] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-17: Connected to tier2-client-17, attached to remote volume '/gluster/mnt6/brick'. [2018-10-05 23:48:36.187109] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-17: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.187155] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-11: Connected to tier2-client-11, attached to remote volume '/gluster/mnt4/brick'. [2018-10-05 23:48:36.187164] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-8: Connected to tier2-client-8, attached to remote volume '/gluster/mnt3/brick'. [2018-10-05 23:48:36.187189] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-8: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.187191] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-11: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.187238] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-20: Connected to tier2-client-20, attached to remote volume '/gluster/mnt7/brick'. [2018-10-05 23:48:36.187260] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-20: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.187263] I [MSGID: 122061] [ec.c:343:ec_up] 0-tier2-disperse-1: Going UP [2018-10-05 23:48:36.187516] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-14: Connected to tier2-client-14, attached to remote volume '/gluster/mnt5/brick'. [2018-10-05 23:48:36.187533] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-14: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.187536] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-23: Connected to tier2-client-23, attached to remote volume '/gluster/mnt8/brick'. [2018-10-05 23:48:36.187562] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-23: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.187569] I [MSGID: 122061] [ec.c:343:ec_up] 0-tier2-disperse-2: Going UP [2018-10-05 23:48:36.187849] I [MSGID: 122061] [ec.c:343:ec_up] 0-tier2-disperse-3: Going UP [2018-10-05 23:48:36.187858] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-10: Server lk version = 1 [2018-10-05 23:48:36.187919] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-36: Connected to tier2-client-36, attached to remote volume '/gluster/mnt1/brick'. [2018-10-05 23:48:36.187939] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-36: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.188183] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-26: Connected to tier2-client-26, attached to remote volume '/gluster/mnt9/brick'. [2018-10-05 23:48:36.188213] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-26: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.188186] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-29: Connected to tier2-client-29, attached to remote volume '/gluster/mnt10/brick'. [2018-10-05 23:48:36.188245] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-29: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.188331] I [MSGID: 122061] [ec.c:343:ec_up] 0-tier2-disperse-4: Going UP [2018-10-05 23:48:36.188493] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-39: Connected to tier2-client-39, attached to remote volume '/gluster/mnt2/brick'. [2018-10-05 23:48:36.188520] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-39: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.188751] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-42: Connected to tier2-client-42, attached to remote volume '/gluster/mnt3/brick'. [2018-10-05 23:48:36.188770] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-42: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.188774] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-32: Connected to tier2-client-32, attached to remote volume '/gluster/mnt11/brick'. [2018-10-05 23:48:36.188808] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-32: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.188820] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-45: Connected to tier2-client-45, attached to remote volume '/gluster/mnt4/brick'. [2018-10-05 23:48:36.188844] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-45: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.188900] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-48: Connected to tier2-client-48, attached to remote volume '/gluster/mnt5/brick'. [2018-10-05 23:48:36.188912] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-48: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.188916] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-37: Connected to tier2-client-37, attached to remote volume '/gluster/mnt1/brick'. [2018-10-05 23:48:36.188938] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-37: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.188966] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-54: Connected to tier2-client-54, attached to remote volume '/gluster/mnt7/brick'. [2018-10-05 23:48:36.188986] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-54: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.189059] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-57: Connected to tier2-client-57, attached to remote volume '/gluster/mnt8/brick'. [2018-10-05 23:48:36.189053] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-51: Connected to tier2-client-51, attached to remote volume '/gluster/mnt6/brick'. [2018-10-05 23:48:36.189073] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-57: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.189092] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-51: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.189189] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-43: Connected to tier2-client-43, attached to remote volume '/gluster/mnt3/brick'. [2018-10-05 23:48:36.189204] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-43: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.189228] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-40: Connected to tier2-client-40, attached to remote volume '/gluster/mnt2/brick'. [2018-10-05 23:48:36.189246] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-40: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.189271] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-60: Connected to tier2-client-60, attached to remote volume '/gluster/mnt9/brick'. [2018-10-05 23:48:36.189286] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-60: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.189314] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-49: Connected to tier2-client-49, attached to remote volume '/gluster/mnt5/brick'. [2018-10-05 23:48:36.189333] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-49: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.189336] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-52: Connected to tier2-client-52, attached to remote volume '/gluster/mnt6/brick'. [2018-10-05 23:48:36.189340] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-46: Connected to tier2-client-46, attached to remote volume '/gluster/mnt4/brick'. [2018-10-05 23:48:36.189376] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-52: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.189406] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-46: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.189417] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-63: Connected to tier2-client-63, attached to remote volume '/gluster/mnt10/brick'. [2018-10-05 23:48:36.189448] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-63: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.189455] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-55: Connected to tier2-client-55, attached to remote volume '/gluster/mnt7/brick'. [2018-10-05 23:48:36.189469] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-55: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.189506] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-66: Connected to tier2-client-66, attached to remote volume '/gluster/mnt11/brick'. [2018-10-05 23:48:36.189528] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-66: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.189559] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-61: Connected to tier2-client-61, attached to remote volume '/gluster/mnt9/brick'. [2018-10-05 23:48:36.189542] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-67: Connected to tier2-client-67, attached to remote volume '/gluster/mnt11/brick'. [2018-10-05 23:48:36.189575] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-61: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.189590] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-67: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.189618] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-13: Server lk version = 1 [2018-10-05 23:48:36.189687] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-64: Connected to tier2-client-64, attached to remote volume '/gluster/mnt10/brick'. [2018-10-05 23:48:36.189697] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-19: Server lk version = 1 [2018-10-05 23:48:36.189694] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-16: Server lk version = 1 [2018-10-05 23:48:36.189724] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-64: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.189755] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-22: Server lk version = 1 [2018-10-05 23:48:36.189758] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-58: Connected to tier2-client-58, attached to remote volume '/gluster/mnt8/brick'. [2018-10-05 23:48:36.189809] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-58: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.189814] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-38: Connected to tier2-client-38, attached to remote volume '/gluster/mnt1/brick'. [2018-10-05 23:48:36.189837] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-38: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.189840] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-28: Server lk version = 1 [2018-10-05 23:48:36.189892] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-47: Connected to tier2-client-47, attached to remote volume '/gluster/mnt4/brick'. [2018-10-05 23:48:36.189911] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-47: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.189917] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-25: Server lk version = 1 [2018-10-05 23:48:36.189976] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-44: Connected to tier2-client-44, attached to remote volume '/gluster/mnt3/brick'. [2018-10-05 23:48:36.189993] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-44: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.190000] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-5: Server lk version = 1 [2018-10-05 23:48:36.190045] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-2: Server lk version = 1 [2018-10-05 23:48:36.190052] I [MSGID: 122061] [ec.c:343:ec_up] 0-tier2-disperse-7: Going UP [2018-10-05 23:48:36.190086] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-50: Connected to tier2-client-50, attached to remote volume '/gluster/mnt5/brick'. [2018-10-05 23:48:36.190105] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-50: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.190309] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-31: Server lk version = 1 [2018-10-05 23:48:36.190351] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-53: Connected to tier2-client-53, attached to remote volume '/gluster/mnt6/brick'. [2018-10-05 23:48:36.190356] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-56: Connected to tier2-client-56, attached to remote volume '/gluster/mnt7/brick'. [2018-10-05 23:48:36.190395] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-56: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.190377] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-53: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.190456] I [MSGID: 122061] [ec.c:343:ec_up] 0-tier2-disperse-8: Going UP [2018-10-05 23:48:36.190481] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-59: Connected to tier2-client-59, attached to remote volume '/gluster/mnt8/brick'. [2018-10-05 23:48:36.190501] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-59: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.190576] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-41: Connected to tier2-client-41, attached to remote volume '/gluster/mnt2/brick'. [2018-10-05 23:48:36.190599] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-41: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.190701] I [MSGID: 122061] [ec.c:343:ec_up] 0-tier2-disperse-9: Going UP [2018-10-05 23:48:36.190713] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-17: Server lk version = 1 [2018-10-05 23:48:36.190857] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-8: Server lk version = 1 [2018-10-05 23:48:36.190887] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-11: Server lk version = 1 [2018-10-05 23:48:36.190913] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-20: Server lk version = 1 [2018-10-05 23:48:36.190940] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-14: Server lk version = 1 [2018-10-05 23:48:36.190958] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-23: Server lk version = 1 [2018-10-05 23:48:36.190966] I [MSGID: 122061] [ec.c:343:ec_up] 0-tier2-disperse-6: Going UP [2018-10-05 23:48:36.190996] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-29: Server lk version = 1 [2018-10-05 23:48:36.191021] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-39: Server lk version = 1 [2018-10-05 23:48:36.191044] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-42: Server lk version = 1 [2018-10-05 23:48:36.191062] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-32: Server lk version = 1 [2018-10-05 23:48:36.191083] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-37: Server lk version = 1 [2018-10-05 23:48:36.191096] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-48: Server lk version = 1 [2018-10-05 23:48:36.191107] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-54: Server lk version = 1 [2018-10-05 23:48:36.191120] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-57: Server lk version = 1 [2018-10-05 23:48:36.191132] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-51: Server lk version = 1 [2018-10-05 23:48:36.191149] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-62: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.191189] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-36: Server lk version = 1 [2018-10-05 23:48:36.191203] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-43: Server lk version = 1 [2018-10-05 23:48:36.191285] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-40: Server lk version = 1 [2018-10-05 23:48:36.204065] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-60: Server lk version = 1 [2018-10-05 23:48:36.204075] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-49: Server lk version = 1 [2018-10-05 23:48:36.204124] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-52: Server lk version = 1 [2018-10-05 23:48:36.204141] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-55: Server lk version = 1 [2018-10-05 23:48:36.204160] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-46: Server lk version = 1 [2018-10-05 23:48:36.204157] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-63: Server lk version = 1 [2018-10-05 23:48:36.204194] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-66: Server lk version = 1 [2018-10-05 23:48:36.204199] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-67: Server lk version = 1 [2018-10-05 23:48:36.204201] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-61: Server lk version = 1 [2018-10-05 23:48:36.204224] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-64: Server lk version = 1 [2018-10-05 23:48:36.204233] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-58: Server lk version = 1 [2018-10-05 23:48:36.204254] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-47: Server lk version = 1 [2018-10-05 23:48:36.204257] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-38: Server lk version = 1 [2018-10-05 23:48:36.204284] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-44: Server lk version = 1 [2018-10-05 23:48:36.204292] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-56: Server lk version = 1 [2018-10-05 23:48:36.204287] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-50: Server lk version = 1 [2018-10-05 23:48:36.204293] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-53: Server lk version = 1 [2018-10-05 23:48:36.204319] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-41: Server lk version = 1 [2018-10-05 23:48:36.204357] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-26: Server lk version = 1 [2018-10-05 23:48:36.204370] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-45: Server lk version = 1 [2018-10-05 23:48:36.204403] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-65: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.204421] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-62: Connected to tier2-client-62, attached to remote volume '/gluster/mnt9/brick'. [2018-10-05 23:48:36.204438] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-62: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.204695] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-59: Server lk version = 1 [2018-10-05 23:48:36.204794] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-68: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:36.204841] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-62: Server lk version = 1 [2018-10-05 23:48:36.205010] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-65: Connected to tier2-client-65, attached to remote volume '/gluster/mnt10/brick'. [2018-10-05 23:48:36.205040] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-65: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.205093] I [MSGID: 122061] [ec.c:343:ec_up] 0-tier2-disperse-10: Going UP [2018-10-05 23:48:36.205183] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-65: Server lk version = 1 [2018-10-05 23:48:36.205319] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-68: Connected to tier2-client-68, attached to remote volume '/gluster/mnt11/brick'. [2018-10-05 23:48:36.205346] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-68: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:36.205472] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-68: Server lk version = 1 [2018-10-05 23:48:37.633386] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-34: changing port to 49163 (from 0) [2018-10-05 23:48:37.635973] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-34: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:37.636540] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-34: Connected to tier2-client-34, attached to remote volume '/gluster/mnt12/brick'. [2018-10-05 23:48:37.636567] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-34: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:37.636680] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-34: Server lk version = 1 [2018-10-05 23:48:37.641303] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-35: changing port to 49163 (from 0) [2018-10-05 23:48:37.643555] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-35: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:37.644194] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-35: Connected to tier2-client-35, attached to remote volume '/gluster/mnt12/brick'. [2018-10-05 23:48:37.644212] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-35: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:37.644268] I [MSGID: 122061] [ec.c:343:ec_up] 0-tier2-disperse-5: Going UP [2018-10-05 23:48:37.644334] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-35: Server lk version = 1 [2018-10-05 23:48:37.705342] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-69: changing port to 49163 (from 0) [2018-10-05 23:48:37.707723] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-69: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:37.708280] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-69: Connected to tier2-client-69, attached to remote volume '/gluster/mnt12/brick'. [2018-10-05 23:48:37.708299] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-69: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:37.708448] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-69: Server lk version = 1 [2018-10-05 23:48:37.713329] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-71: changing port to 49163 (from 0) [2018-10-05 23:48:37.715794] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-71: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:37.716353] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-71: Connected to tier2-client-71, attached to remote volume '/gluster/mnt12/brick'. [2018-10-05 23:48:37.716370] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-71: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:37.716485] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-71: Server lk version = 1 [2018-10-05 23:48:38.641515] I [rpc-clnt.c:1986:rpc_clnt_reconfig] 0-tier2-client-70: changing port to 49163 (from 0) [2018-10-05 23:48:38.643920] I [MSGID: 114057] [client-handshake.c:1478:select_server_supported_programs] 0-tier2-client-70: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2018-10-05 23:48:38.644811] I [MSGID: 114046] [client-handshake.c:1231:client_setvolume_cbk] 0-tier2-client-70: Connected to tier2-client-70, attached to remote volume '/gluster/mnt12/brick'. [2018-10-05 23:48:38.644828] I [MSGID: 114047] [client-handshake.c:1242:client_setvolume_cbk] 0-tier2-client-70: Server and Client lk-version numbers are not same, reopening the fds [2018-10-05 23:48:38.644896] I [MSGID: 122061] [ec.c:343:ec_up] 0-tier2-disperse-11: Going UP [2018-10-05 23:48:38.644978] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 0-tier2-client-70: Server lk version = 1 [2018-10-05 23:48:44.735323] I [dht-rebalance.c:4512:gf_defrag_start_crawl] 0-tier2-dht: gf_defrag_start_crawl using commit hash 3720331860 [2018-10-05 23:48:44.736205] W [MSGID: 122040] [ec-common.c:1097:ec_prepare_update_cbk] 0-tier2-disperse-7: Failed to get size and version [Input/output error] [2018-10-05 23:48:44.736266] E [MSGID: 122034] [ec-common.c:613:ec_child_select] 0-tier2-disperse-7: Insufficient available children for this request (have 0, need 4) [2018-10-05 23:48:44.736282] E [MSGID: 122037] [ec-common.c:2040:ec_update_size_version_done] 0-tier2-disperse-7: Failed to update version and size [Input/output error] [2018-10-05 23:48:44.736377] W [MSGID: 122040] [ec-common.c:1097:ec_prepare_update_cbk] 0-tier2-disperse-8: Failed to get size and version [Input/output error] [2018-10-05 23:48:44.736436] E [MSGID: 122034] [ec-common.c:613:ec_child_select] 0-tier2-disperse-8: Insufficient available children for this request (have 0, need 4) [2018-10-05 23:48:44.736459] E [MSGID: 122037] [ec-common.c:2040:ec_update_size_version_done] 0-tier2-disperse-8: Failed to update version and size [Input/output error] [2018-10-05 23:48:44.736460] W [MSGID: 122040] [ec-common.c:1097:ec_prepare_update_cbk] 0-tier2-disperse-10: Failed to get size and version [Input/output error] [2018-10-05 23:48:44.736537] W [MSGID: 122040] [ec-common.c:1097:ec_prepare_update_cbk] 0-tier2-disperse-9: Failed to get size and version [Input/output error] [2018-10-05 23:48:44.736571] E [MSGID: 122034] [ec-common.c:613:ec_child_select] 0-tier2-disperse-10: Insufficient available children for this request (have 0, need 4) [2018-10-05 23:48:44.736574] E [MSGID: 122034] [ec-common.c:613:ec_child_select] 0-tier2-disperse-9: Insufficient available children for this request (have 0, need 4) [2018-10-05 23:48:44.736604] E [MSGID: 122037] [ec-common.c:2040:ec_update_size_version_done] 0-tier2-disperse-9: Failed to update version and size [Input/output error] [2018-10-05 23:48:44.736604] E [MSGID: 122037] [ec-common.c:2040:ec_update_size_version_done] 0-tier2-disperse-10: Failed to update version and size [Input/output error] [2018-10-05 23:48:44.736827] W [MSGID: 122040] [ec-common.c:1097:ec_prepare_update_cbk] 0-tier2-disperse-11: Failed to get size and version [Input/output error] [2018-10-05 23:48:44.736887] E [MSGID: 122034] [ec-common.c:613:ec_child_select] 0-tier2-disperse-11: Insufficient available children for this request (have 0, need 4) [2018-10-05 23:48:44.736904] E [MSGID: 122037] [ec-common.c:2040:ec_update_size_version_done] 0-tier2-disperse-11: Failed to update version and size [Input/output error] [2018-10-05 23:48:44.740337] W [MSGID: 122040] [ec-common.c:1097:ec_prepare_update_cbk] 0-tier2-disperse-6: Failed to get size and version [Input/output error] [2018-10-05 23:48:44.740381] E [MSGID: 122034] [ec-common.c:613:ec_child_select] 0-tier2-disperse-6: Insufficient available children for this request (have 0, need 4) [2018-10-05 23:48:44.740394] E [MSGID: 122037] [ec-common.c:2040:ec_update_size_version_done] 0-tier2-disperse-6: Failed to update version and size [Input/output error] [2018-10-05 23:48:50.066103] I [MSGID: 109081] [dht-common.c:4379:dht_setxattr] 0-tier2-dht: fixing the layout of /
|
_______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx https://lists.gluster.org/mailman/listinfo/gluster-users
