[Sorry for replying to multiple mails in the same thread] ----- Original Message ----- > I have two production machines (web1, web2) that are currently using > Glusterfs. I added two new machines, web3 and web4. Web1 and web2 are peered > and are running great. Web3 and web4 will peer with each other but web1 and > web2 will not peer with web3 or web4 and vice versa. Did you add web3 and web4 using the following procedure? from web1 (or web2), web1# gluster peer probe web3 web1# gluster peer probe web4 Only a peer that is (already) part of a cluster can add a new peer to the (same) cluster. i.e a peer cannot seek membership into the cluster, only a member can 'invite' a new peer. HTH, kp > > All machines are running Gluster 3.6.1: > ii glusterfs-client 3.6.1-1 amd64 clustered file-system (client package) > ii glusterfs-common 3.6.1-1 amd64 GlusterFS common libraries and translator > modules > ii glusterfs-server 3.6.1-1 amd64 clustered file-system (server package) > > I ran "gluster volume set all cluster.op-version 30600" on web2, now all four > machines have "operating-version=30600" in /var/lib/glusterd/ glusterd.info > > All machines have the same kernel: > Linux web1 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt2-1 (2014-12-08) x86_64 > GNU/Linux > > /etc/hosts is managed with Ansible, all machines resolve by their names > (web1, web2, web3, web4) to their internal IP. > > "hping3 -8 111,2049,24007,49152-49162 -S [web1-web4]" shows all Gluster ports > open from all servers. > > root@web1:~# gluster peer probe web2 > peer probe: success. Host web2 port 24007 already in peer list > root@web1:~# gluster peer probe web3 > peer probe: failed: Probe returned with unknown errno 107 > root@web1:~# gluster peer probe web4 > peer probe: failed: Probe returned with unknown errno 107 > > > root@web3:~# gluster peer probe web1 > peer probe: failed: Probe returned with unknown errno 107 > root@web3:~# gluster peer probe web2 > peer probe: failed: Probe returned with unknown errno 107 > root@web3:~# gluster peer probe web4 > peer probe: success. Host web4 port 24007 already in peer list > > > from /var/log/glusterfs/cli.log: > > [2015-01-07 20:38:33.978304] T [cli.c:264:cli_rpc_notify] 0-glusterfs: got > RPC_CLNT_CONNECT > [2015-01-07 20:38:33.978322] T [cli-quotad-client.c:94:cli_quotad_notify] > 0-glusterfs: got RPC_CLNT_CONNECT > [2015-01-07 20:38:33.978333] I [socket.c:2344:socket_event_handler] > 0-transport: disconnecting now > [2015-01-07 20:38:33.978354] T [cli-quotad-client.c:100:cli_quotad_notify] > 0-glusterfs: got RPC_CLNT_DISCONNECT > [2015-01-07 20:38:33.978622] T [rpc-clnt.c:1381:rpc_clnt_record] 0-glusterfs: > Auth Info: pid: 0, uid: 0, gid: 0, owner: > [2015-01-07 20:38:33.978681] T [rpc-clnt.c:1238:rpc_clnt_record_build_header] > 0-rpc-clnt: Request fraglen 144, payload: 80, rpc hdr: 64 > [2015-01-07 20:38:33.979037] T [socket.c:2863:socket_connect] (--> > /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(_gf_log_callingfn+0x1cd)[0x7f9e8ef0ac1d] > (--> > /usr/lib/x86_64-linux-gnu/glusterfs/3.6.1/rpc-transport/socket.so(+0x637c)[0x7f9e8bcc637c] > (--> > /usr/lib/x86_64-linux-gnu/libgfrpc.so.0(rpc_clnt_submit+0x3a4)[0x7f9e8e40fad4] > (--> gluster(cli_submit_request+0x15f)[0x7f9e8f3c52df] (--> > gluster(cli_cmd_submit+0x8b)[0x7f9e8f3c6eeb] ))))) 0-glusterfs: connect () > called on transport already connected > [2015-01-07 20:38:33.979100] T [rpc-clnt.c:1573:rpc_clnt_submit] 0-rpc-clnt: > submitted request (XID: 0x1 Program: Gluster CLI, ProgVers: 2, Proc: 1) to > rpc-transport (glusterfs) > [2015-01-07 20:38:33.979111] D [rpc-clnt-ping.c:231:rpc_clnt_start_ping] > 0-glusterfs: ping timeout is 0, returning > [2015-01-07 20:38:33.986269] T [rpc-clnt.c:660:rpc_clnt_reply_init] > 0-glusterfs: received rpc message (RPC XID: 0x1 Program: Gluster CLI, > ProgVers: 2, Proc: 1) from rpc-transport (glusterfs) > [2015-01-07 20:38:33.986311] I [cli-rpc-ops.c:131:gf_cli_probe_cbk] 0-cli: > Received resp to probe > [2015-01-07 20:38:33.986321] E [cli-rpc-ops.c:136:gf_cli_probe_cbk] 0-cli: > Probe returned with unknown errno 107 > [2015-01-07 20:38:33.986620] D [cli-cmd.c:384:cli_cmd_submit] 0-cli: > Returning -1 > [2015-01-07 20:38:33.986638] D [cli-rpc-ops.c:3021:gf_cli_probe] 0-cli: > Returning -1 > [2015-01-07 20:38:33.986650] D [cli-cmd-peer.c:96:cli_cmd_peer_probe_cbk] > 0-cli: frame->local is not NULL (0x7f9e7c000980) > [2015-01-07 20:38:33.986667] I [input.c:36:cli_batch] 0-: Exiting with: -1 > > > > Thanks in advance for any help. > > > > _______________________________________________ > Gluster-users mailing list > Gluster-users@xxxxxxxxxxx > http://www.gluster.org/mailman/listinfo/gluster-users _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-users
