Hi everybody!
I have "gluster peer probe gfs1" command hung with the result of "Probe
Sent to Peer (connected)"
gfs3#gluster peer status
Number of Peers: 3
Hostname: gfs6
Uuid: 6bd6ee25-e257-4703-b500-330741b90471
State: Peer in Cluster (Connected)
Hostname: gfs4
Uuid: bb1bed20-25bf-43b0-8faa-49f1b5b9ae59
State: Peer in Cluster (Connected)
Hostname: gfs1
Uuid: bb67c1da-2698-4c35-b29d-f80f8eb814a6
State: Probe Sent to Peer (Connected)
I double checked the dns resolution. Forward and reverse resolution
works fine.
I tried to debug on gfs1 and gfs3 at the moment of probe command and it
seems to me that gfs1 does not sync all the gluster configuration from
gfs3. Especially peers folder contains only 1 peer with the following
content:
gfs1#cat /var/lib/glusterd/peers/192.168.9.53
uuid=00000000-0000-0000-0000-000000000000
state=8
hostname1=192.168.9.53
so there are no uuids both in the content of the file and in filename
itself. Besides there no info about other peers like gfs4, gfs6 in the
folder.
Last lines of the debug log on gfs1 looks like this:
[2015-04-23 10:12:28.334239] D [common-utils.c:2930:gf_is_local_addr]
0-management: 192.168.9.54
[2015-04-23 10:12:28.334413] D [common-utils.c:2930:gf_is_local_addr]
0-management: 192.168.9.54
[2015-04-23 10:12:28.334557] D [common-utils.c:2930:gf_is_local_addr]
0-management: 192.168.9.54
[2015-04-23 10:12:28.334697] D [common-utils.c:2946:gf_is_local_addr]
0-management: gfs4 is not local
[2015-04-23 10:12:28.334707] D
[glusterd-utils.c:5567:glusterd_hostname_to_uuid] 0-management:
returning -1
[2015-04-23 10:12:28.334714] D
[glusterd-utils.c:1035:glusterd_volume_brickinfo_get] 0-management:
Returning -1
[2015-04-23 10:12:28.336881] D
[glusterd-utils.c:5532:glusterd_friend_find_by_hostname] 0-management:
Unable to find friend: gfs6
[2015-04-23 10:12:28.337490] D [common-utils.c:2930:gf_is_local_addr]
0-management: 192.168.9.56
[2015-04-23 10:12:28.337697] D [common-utils.c:2930:gf_is_local_addr]
0-management: 192.168.9.56
[2015-04-23 10:12:28.337841] D [common-utils.c:2930:gf_is_local_addr]
0-management: 192.168.9.56
[2015-04-23 10:12:28.337981] D [common-utils.c:2946:gf_is_local_addr]
0-management: gfs6 is not local
[2015-04-23 10:12:28.337991] D
[glusterd-utils.c:5567:glusterd_hostname_to_uuid] 0-management:
returning -1
[2015-04-23 10:12:28.337998] D
[glusterd-utils.c:1035:glusterd_volume_brickinfo_get] 0-management:
Returning -1
[2015-04-23 10:12:28.338858] D
[glusterd-utils.c:5523:glusterd_friend_find_by_hostname] 0-management:
Friend gfs3 found.. state: 8
[2015-04-23 10:12:28.338873] D
[glusterd-utils.c:5567:glusterd_hostname_to_uuid] 0-management:
returning 0
[2015-04-23 10:12:28.340989] D
[glusterd-utils.c:5532:glusterd_friend_find_by_hostname] 0-management:
Unable to find friend: gfs6
[2015-04-23 10:12:28.341545] D [common-utils.c:2930:gf_is_local_addr]
0-management: 192.168.9.56
[2015-04-23 10:12:28.341697] D [common-utils.c:2930:gf_is_local_addr]
0-management: 192.168.9.56
[2015-04-23 10:12:28.341840] D [common-utils.c:2930:gf_is_local_addr]
0-management: 192.168.9.56
[2015-04-23 10:12:28.341980] D [common-utils.c:2946:gf_is_local_addr]
0-management: gfs6 is not local
[2015-04-23 10:12:28.341991] D
[glusterd-utils.c:5567:glusterd_hostname_to_uuid] 0-management:
returning -1
[2015-04-23 10:12:28.341997] D
[glusterd-utils.c:685:glusterd_resolve_brick] 0-management: Returning -1
[2015-04-23 10:12:28.342003] D
[glusterd-utils.c:1035:glusterd_volume_brickinfo_get] 0-management:
Returning -1
[2015-04-23 10:12:28.344065] D
[glusterd-utils.c:5532:glusterd_friend_find_by_hostname] 0-management:
Unable to find friend: gfs4
[2015-04-23 10:12:28.344620] D [common-utils.c:2930:gf_is_local_addr]
0-management: 192.168.9.54
[2015-04-23 10:12:28.344772] D [common-utils.c:2930:gf_is_local_addr]
0-management: 192.168.9.54
[2015-04-23 10:12:28.344914] D [common-utils.c:2930:gf_is_local_addr]
0-management: 192.168.9.54
[2015-04-23 10:12:28.345054] D [common-utils.c:2946:gf_is_local_addr]
0-management: gfs4 is not local
[2015-04-23 10:12:28.345064] D
[glusterd-utils.c:5567:glusterd_hostname_to_uuid] 0-management:
returning -1
[2015-04-23 10:12:28.345071] D
[glusterd-utils.c:1035:glusterd_volume_brickinfo_get] 0-management:
Returning -1
[2015-04-23 10:12:28.345543] D [run.c:190:runner_log] 0-: Starting the
nfs/glustershd services: /usr/sbin/glusterfs -s localhost --volfile-id
gluster/quotad -p /var/lib/glusterd/quotad/run/quotad.pid -l
/var/log/glusterfs/quotad.log -S
/var/run/3e619fbfe69c96b1dbc7486a7d38a7be.socket --xlator-option
*replicate*.data-self-heal=off --xlator-option
*replicate*.metadata-self-heal=off --xlator-option
*replicate*.entry-self-heal=off
^C[2015-04-23 10:20:50.237403] W [glusterfsd.c:1095:cleanup_and_exit]
(-->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(runner_end_reuse+0x26)
[0x7fa81cd3dfb6]
(-->/lib/x86_64-linux-gnu/libpthread.so.0(waitpid+0x5b)
[0x7fa81c499c8b] (-->/lib/x86_64-linux-gnu/libc.so.6(+0x321e0)
[0x7fa81bd3a1e0]))) 0-: received signum (2), shutting down
[2015-04-23 10:20:50.237436] D
[glusterfsd-mgmt.c:2025:glusterfs_mgmt_pmap_signout] 0-fsd-mgmt:
portmapper signout arguments not given
So I think server hangs when trying to start volume with references to
gfs4 and gfs6 which it does not know about, but why gfs1 peers
configuration is empty?
Any help?
Thanks in advance!
Best regards,
Alex.
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users
