I think your setup is messed up. peer status from these two nodes are completely out of sync. Another thumb-rule related to peer probe is you should probe a node from your existing cluster not the vice versa. I'd request you to clean up /var/lib/glusterd/* from the faulty node, restart glusterd service and then attempt to probe this node from either main/glusternode2/glusternode4. That should work. Thanks, Atin On 11/27/2015 12:01 PM, Frank Rothenstein wrote: > Hi Atin, > > my problems also started by not connecting correctly, so I followed > advices to reconnect by deleting contents of /var/lib/gluster, then > peer probe an sync. It syncs ok, the "main" hosts peers correctly with > another one, which has no bricks atm: > > main: > gluster peer status > Number of Peers: 2 > > Hostname: glusternode2 > Uuid: 5fd7c455-4eef-4e1f-9ef4-91b47748bebc > State: Sent and Received peer request (Connected) > Other names: > 192.168.222.101 > 139.64.206.101 > > Hostname: glusternode4 > Uuid: 4a7bd9b7-40d2-49ba-8ec6-eefcf46b6e5b > State: Peer in Cluster (Connected) > Other names: > 139.64.206.103 > 192.168.222.103 > > faulty: > > gluster peer probe glusternode4 > peer probe: failed: Probe returned with Transport endpoint is not > connected > > gluster peer status > Number of Peers: 1 > > Hostname: glusternode1 > Uuid: 56a2baab-6e33-42e7-9d4d-f5fe3c9fcdd4 > State: Probe Sent to Peer (Connected) > Other names: > 139.64.206.100 > 192.168.206.100 > > What I don't understand is, why no .socket files are created in > /var/run/gluster and rpcinfo shows no registered services... > > When I started the NFS-Kernel-Server for testing purposes these > services got registered... > > Enough space left, matching permissions, same versions as on the other > host - I'm still stuck. > > Thanks, Frank > > Am Donnerstag, den 26.11.2015, 22:12 +0530 schrieb Atin Mukherjee: >> The logs seems to be normal, do you see any difference in peer status >> command output from different nodes? >> -Atin >> Sent from one plus one >> On Nov 26, 2015 9:04 PM, "Frank Rothenstein" <f.rothenstein@bodden-kl >> iniken.de> wrote: >>> Today's cli.log: http://pastebin.com/MjbxRkTj >>> Last start of glusterd.service, etc-glusterfs- >>> glusterd.vol.log: http:// >>> pastebin.com/21gDJZwb >>> A lot of output from glustershd.log: http://pastebin.com/rTXMqTPC >>> >>> >>> Am Donnerstag, den 26.11.2015, 20:41 +0530 schrieb Atin Mukherjee: >>>> Please attach the glusterd log of the faulty node. >>>> -Atin >>>> Sent from one plus one >>>> On Nov 26, 2015 8:31 PM, "Frank Rothenstein" <f.rothenstein@bodde >>> n-kl >>>> iniken.de> wrote: >>>>> Hey @all, >>>>> >>>>> I'm having trouble on one of my gluster hosts/peers: >>>>> glusterd is starting but won't connect properly to other >>> peer(s). >>>>> >>>>> I'm stuck, after reading Bugzilla/Google Groups/Gluster Users >>> etc I >>>>> still have no clue why Gluster won't start only on this >>> particular >>>>> machine. It's Centos 7, i have another machine (same hardware, >>> same >>>>> config) running with no problems. >>>>> The weirdest part is that the cluster ran before and since the >>>>> upgrade >>>>> with only minor problems. Then i had to restart the machines >>> and >>>>> this >>>>> one never came back properly. >>>>> I'm can't find the cause and appreciate any help. >>>>> >>>>> rpcinfo -p >>>>> program vers proto port service >>>>> 100000 4 tcp 111 portmapper >>>>> 100000 3 tcp 111 portmapper >>>>> 100000 2 tcp 111 portmapper >>>>> 100000 4 udp 111 portmapper >>>>> 100000 3 udp 111 portmapper >>>>> 100000 2 udp 111 portmapper >>>>> 100024 1 udp 48184 status >>>>> 100024 1 tcp 34779 status >>>>> >>>>> Then of course: >>>>> >>>>> showmount -e >>>>> clnt_create: RPC: Program not registered >>>>> >>>>> >>>>> No .socket-files >>>>> >>>>> ls /var/run/gluster >>>>> snaps >>>>> >>>>> If I run glusterd in DEBUG I'm constantly getting: >>>>> >>>>> [2015-11-26 14:30:05.290866] D [socket.c:558:__socket_rwv] 0- >>>>> socket.management: EOF on socket >>>>> [2015-11-26 14:30:05.290886] D >>> [socket.c:2355:socket_event_handler] >>>>> 0- >>>>> transport: disconnecting now >>>>> [2015-11-26 14:30:05.290903] D [MSGID: 0] [event- >>>>> epoll.c:591:event_dispatch_epoll_handler] 0-epoll: generation >>>>> bumped on >>>>> idx=4 from gen=382 to slot->gen=383, fd=10, slot->fd=10 >>>>> >>>>> >>>>> ls -al /var/run/gluster >>>>> total 0 >>>>> drwxr-xr-x 3 root root 60 Nov 26 12:51 . >>>>> drwxr-xr-x 37 root root 1180 Nov 26 15:30 .. >>>>> drwxr-xr-x 2 root root 40 Nov 26 12:51 snaps >>>>> >>>>> Thanks in advance >>>>> >>>>> Frank >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>> ___________________________________________________________________ >>>>> ___________ >>>>> BODDEN-KLINIKEN Ribnitz-Damgarten GmbH >>>>> Sandhufe 2 >>>>> 18311 Ribnitz-Damgarten >>>>> >>>>> Telefon: 03821-700-0 >>>>> Fax: 03821-700-240 >>>>> >>>>> E-Mail: info@xxxxxxxxxxxxxxxxxx Internet: http://www.bodden-k >>> lini >>>>> ken.de >>>>> >>>>> Sitz: Ribnitz-Damgarten, Amtsgericht: Stralsund, HRB 2919, >>> Steuer- >>>>> Nr.: 079/133/40188 >>>>> Aufsichtsratsvorsitzende: Carmen Schröter, Geschäftsführer: Dr. >>>>> Falko Milski >>>>> >>>>> Der Inhalt dieser E-Mail ist ausschließlich für den >>> bezeichneten >>>>> Adressaten bestimmt. Wenn Sie nicht der vorge- >>>>> sehene Adressat dieser E-Mail oder dessen Vertreter sein >>> sollten, >>>>> beachten Sie bitte, dass jede Form der Veröf- >>>>> fentlichung, Vervielfältigung oder Weitergabe des Inhalts >>> dieser E- >>>>> Mail unzulässig ist. Wir bitten Sie, sofort den >>>>> Absender zu informieren und die E-Mail zu löschen. >>>>> >>>>> >>>>> Bodden-Kliniken Ribnitz-Damgarten GmbH 2015 >>>>> *** Virenfrei durch Kerio Mail Server und Sophos Antivirus *** >>>>> >>>>> _______________________________________________ >>>>> Gluster-users mailing list >>>>> Gluster-users@xxxxxxxxxxx >>>>> http://www.gluster.org/mailman/listinfo/gluster-users >>> >>> >>> >>> >>> >>> ___________________________________________________________________ >>> ___________ >>> BODDEN-KLINIKEN Ribnitz-Damgarten GmbH >>> Sandhufe 2 >>> 18311 Ribnitz-Damgarten >>> >>> Telefon: 03821-700-0 >>> Fax: 03821-700-240 >>> >>> E-Mail: info@xxxxxxxxxxxxxxxxxx Internet: http://www.bodden-klini >>> ken.de >>> >>> Sitz: Ribnitz-Damgarten, Amtsgericht: Stralsund, HRB 2919, Steuer- >>> Nr.: 079/133/40188 >>> Aufsichtsratsvorsitzende: Carmen Schröter, Geschäftsführer: Dr. >>> Falko Milski >>> >>> Der Inhalt dieser E-Mail ist ausschließlich für den bezeichneten >>> Adressaten bestimmt. Wenn Sie nicht der vorge- >>> sehene Adressat dieser E-Mail oder dessen Vertreter sein sollten, >>> beachten Sie bitte, dass jede Form der Veröf- >>> fentlichung, Vervielfältigung oder Weitergabe des Inhalts dieser E- >>> Mail unzulässig ist. Wir bitten Sie, sofort den >>> Absender zu informieren und die E-Mail zu löschen. >>> >>> >>> Bodden-Kliniken Ribnitz-Damgarten GmbH 2015 >>> *** Virenfrei durch Kerio Mail Server und Sophos Antivirus *** >>> >>> > > > > > > ______________________________________________________________________________ > BODDEN-KLINIKEN Ribnitz-Damgarten GmbH > Sandhufe 2 > 18311 Ribnitz-Damgarten > > Telefon: 03821-700-0 > Fax: 03821-700-240 > > E-Mail: info@xxxxxxxxxxxxxxxxxx Internet: http://www.bodden-kliniken.de > > Sitz: Ribnitz-Damgarten, Amtsgericht: Stralsund, HRB 2919, Steuer-Nr.: 079/133/40188 > Aufsichtsratsvorsitzende: Carmen Schröter, Geschäftsführer: Dr. Falko Milski > > Der Inhalt dieser E-Mail ist ausschließlich für den bezeichneten Adressaten bestimmt. Wenn Sie nicht der vorge- > sehene Adressat dieser E-Mail oder dessen Vertreter sein sollten, beachten Sie bitte, dass jede Form der Veröf- > fentlichung, Vervielfältigung oder Weitergabe des Inhalts dieser E-Mail unzulässig ist. Wir bitten Sie, sofort den > Absender zu informieren und die E-Mail zu löschen. > > > Bodden-Kliniken Ribnitz-Damgarten GmbH 2015 > *** Virenfrei durch Kerio Mail Server und Sophos Antivirus *** > > _______________________________________________ > Gluster-users mailing list > Gluster-users@xxxxxxxxxxx > http://www.gluster.org/mailman/listinfo/gluster-users > _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-users
