On 04/09/2015 06:21 PM, Pierre Léonard wrote: > Hi all, > > I have a 14 node cluster with two volume on replica 7 one strip 7. > This night the node 10 add a peers files and stop. > I have corrected with the data in glusterd.info and suppress the bad peers file. Could you clarify what steps did you perform here. Also could you try to start glusterd with -LDEBUG and share the glusterd log file with us. Also do you see any delta in glusterd.info file between node 10 and the other nodes? ~Atin > > Now all the node are stopped no glusterd service and the node 10 failed in > starting with the following message : > > [2015-04-09 12:36:34.441595] I [MSGID: 100030] [glusterfsd.c:2018:main] > 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.6.2 (args: > /usr/sbin/glusterd --pid-file=/va\ > r/run/glusterd.pid) > [2015-04-09 12:36:34.447117] I [glusterd.c:1214:init] 0-management: Maximum > allowed open file descriptors set to 65536 > [2015-04-09 12:36:34.447181] I [glusterd.c:1259:init] 0-management: Using > /var/lib/glusterd as working directory > [2015-04-09 12:36:34.452105] W [rdma.c:4221:__gf_rdma_ctx_create] > 0-rpc-transport/rdma: rdma_cm event channel creation failed (No such device) > [2015-04-09 12:36:34.452140] E [rdma.c:4519:init] 0-rdma.management: Failed to > initialize IB Device > [2015-04-09 12:36:34.452156] E [rpc-transport.c:333:rpc_transport_load] > 0-rpc-transport: 'rdma' initialization failed > [2015-04-09 12:36:34.452233] W [rpcsvc.c:1524:rpcsvc_transport_create] > 0-rpc-service: cannot create listener, initing the transport failed > [2015-04-09 12:36:41.418761] I > [glusterd-store.c:2043:glusterd_restore_op_version] 0-glusterd: retrieved > op-version: 2 > [2015-04-09 12:36:42.107207] I > [glusterd-handler.c:3146:glusterd_friend_add_from_peerinfo] 0-management: > connect returned 0 > [2015-04-09 12:36:42.118716] I > [glusterd-handler.c:3146:glusterd_friend_add_from_peerinfo] 0-management: > connect returned 0 > [2015-04-09 12:36:42.130187] I > [glusterd-handler.c:3146:glusterd_friend_add_from_peerinfo] 0-management: > connect returned 0 > [2015-04-09 12:36:42.141720] I > [glusterd-handler.c:3146:glusterd_friend_add_from_peerinfo] 0-management: > connect returned 0 > [2015-04-09 12:36:42.153222] I > [glusterd-handler.c:3146:glusterd_friend_add_from_peerinfo] 0-management: > connect returned 0 > [2015-04-09 12:36:42.164689] I > [glusterd-handler.c:3146:glusterd_friend_add_from_peerinfo] 0-management: > connect returned 0 > [2015-04-09 12:36:42.176217] I > [glusterd-handler.c:3146:glusterd_friend_add_from_peerinfo] 0-management: > connect returned 0 > [2015-04-09 12:36:42.187721] I > [glusterd-handler.c:3146:glusterd_friend_add_from_peerinfo] 0-management: > connect returned 0 > [2015-04-09 12:36:42.199244] I > [glusterd-handler.c:3146:glusterd_friend_add_from_peerinfo] 0-management: > connect returned 0 > [2015-04-09 12:36:42.210729] I > [glusterd-handler.c:3146:glusterd_friend_add_from_peerinfo] 0-management: > connect returned 0 > [2015-04-09 12:36:42.222230] I > [glusterd-handler.c:3146:glusterd_friend_add_from_peerinfo] 0-management: > connect returned 0 > [2015-04-09 12:36:42.233736] I > [glusterd-handler.c:3146:glusterd_friend_add_from_peerinfo] 0-management: > connect returned 0 > [2015-04-09 12:36:42.233867] I [rpc-clnt.c:969:rpc_clnt_connection_init] > 0-management: setting frame-timeout to 600 > [2015-04-09 12:36:42.240940] I [rpc-clnt.c:969:rpc_clnt_connection_init] > 0-management: setting frame-timeout to 600 > [2015-04-09 12:36:42.246986] I [rpc-clnt.c:969:rpc_clnt_connection_init] > 0-management: setting frame-timeout to 600 > [2015-04-09 12:36:42.252953] I [rpc-clnt.c:969:rpc_clnt_connection_init] > 0-management: setting frame-timeout to 600 > [2015-04-09 12:36:42.258897] I [rpc-clnt.c:969:rpc_clnt_connection_init] > 0-management: setting frame-timeout to 600 > [2015-04-09 12:36:42.264829] I [rpc-clnt.c:969:rpc_clnt_connection_init] > 0-management: setting frame-timeout to 600 > [2015-04-09 12:36:42.270775] I [rpc-clnt.c:969:rpc_clnt_connection_init] > 0-management: setting frame-timeout to 600 > [2015-04-09 12:36:42.276730] I [rpc-clnt.c:969:rpc_clnt_connection_init] > 0-management: setting frame-timeout to 600 > [2015-04-09 12:36:42.282703] I [rpc-clnt.c:969:rpc_clnt_connection_init] > 0-management: setting frame-timeout to 600 > [2015-04-09 12:36:42.288624] I [rpc-clnt.c:969:rpc_clnt_connection_init] > 0-management: setting frame-timeout to 600 > [2015-04-09 12:36:42.294571] I [rpc-clnt.c:969:rpc_clnt_connection_init] > 0-management: setting frame-timeout to 600 > [2015-04-09 12:36:42.300498] I [rpc-clnt.c:969:rpc_clnt_connection_init] > 0-management: setting frame-timeout to 600 > [2015-04-09 12:36:42.306460] I > [glusterd-store.c:3497:glusterd_store_retrieve_missed_snaps_list] 0-management: > No missed snaps list. > [2015-04-09 12:36:42.315087] E > [glusterd-store.c:4244:glusterd_resolve_all_bricks] 0-glusterd: resolve brick > failed in restore > [2015-04-09 12:36:42.315145] E [xlator.c:425:xlator_init] 0-management: > Initialization of volume 'management' failed, review your volfile again > [2015-04-09 12:36:42.315168] E [graph.c:322:glusterfs_graph_init] 0-management: > initializing translator failed > [2015-04-09 12:36:42.315183] E [graph.c:525:glusterfs_graph_activate] 0-graph: > init failed > [2015-04-09 12:36:42.315750] W [glusterfsd.c:1194:cleanup_and_exit] (--> 0-: > received signum (0), shutting down > > > So something happen in the local configuration that made the glustr daemon to > failed. I Hav controlled all the peers files, buat with no changes. > The release is le last 3.6.2-1 > Any idea ? > > Many thanks for your help. > > -- > Signature electronique > INRA <http://www.inra.fr> > *Pierre Léonard* > *Senior IT Manager* > *MetaGenoPolis* > Pierre.Leonard@xxxxxxxxxxxx <mailto:Pierre.Leonard@xxxxxxxxxxxx> > Tél. : +33 (0)1 34 65 29 78 > > Centre de recherche INRA > Domaine de Vilvert Bât. 325 R+1 > 78 352 Jouy-en-Josas CEDEX > France > www.mgps.eu <http://www.mgps.eu> > > > > _______________________________________________ > Gluster-users mailing list > Gluster-users@xxxxxxxxxxx > http://www.gluster.org/mailman/listinfo/gluster-users > -- ~Atin _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-users