Re: one peer flooded with - 0-glusterfs: connection attempt on 127.0.0.1:24007 failed, (Invalid argument)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 14/10/2019 08:33, Sanju Rakonde wrote:
please check contents of /var/lib/glusterd/peers/ directory, it should not have any information regarding the localhost. Please check the uuid of the local node at /var/lib/glusterd/glusterd.info file and figure out if you have a file with this uuid at /var/lib/glusterd/peers/*. If you find any such file, please delete it and restart glusterd on that node.

On Fri, Oct 11, 2019 at 3:15 PM lejeczek <peljasz@xxxxxxxxxxx> wrote:
hi guys,

as per the subject.

Only one thing that I'd like to tell first is that on that peer/node
Samba runs. Other two peers do not show this in their logs.

I gluster log for the volume I get plenty of:

...

t)
[2019-10-11 09:40:40.768647] E [socket.c:3498:socket_connect]
0-glusterfs: connection attempt on 127.0.0.1:24007 failed, (Invalid
argument)
[2019-10-11 09:40:43.777129] E [socket.c:3498:socket_connect]
0-glusterfs: connection attempt on 127.0.0.1:24007 failed, (Invalid
argument)
[2019-10-11 09:40:46.785522] E [socket.c:3498:socket_connect]
0-glusterfs: connection attempt on 127.0.0.1:24007 failed, (Invalid
argument)
[2019-10-11 09:40:49.794393] E [socket.c:3498:socket_connect]
0-glusterfs: connection attempt on 127.0.0.1:24007 failed, (Invalid
argument)
[2019-10-11 09:40:52.805158] E [socket.c:3498:socket_connect]
0-glusterfs: connection attempt on 127.0.0.1:24007 failed, (Invalid
argument)
[2019-10-11 09:40:55.817603] E [socket.c:3498:socket_connect]
0-glusterfs: connection attempt on 127.0.0.1:24007 failed, (Invalid
argument)
[2019-10-11 09:40:58.826136] E [socket.c:3498:socket_connect]
0-glusterfs: connection attempt on 127.0.0.1:24007 failed, (Invalid
argument)
[2019-10-11 09:41:01.836104] E [socket.c:3498:socket_connect]
0-glusterfs: connection attempt on 127.0.0.1:24007 failed, (Invalid
argument)
[2019-10-11 09:41:04.842676] E [socket.c:3498:socket_connect]
0-glusterfs: connection attempt on 127.0.0.1:24007 failed, (Invalid
argument)
...

Cluster runs off Centos 7 an ver is 6.5.

glusterd.vol is on all three peers as here:

volume management
    type mgmt/glusterd
    option working-directory /var/lib/glusterd
    option transport-type socket,rdma
    option transport.socket.keepalive-time 10
    option transport.socket.keepalive-interval 2
    option transport.socket.read-fail-log off
    option transport.socket.listen-port 24007
    option transport.rdma.listen-port 24008
    option ping-timeout 0
    option event-threads 1
#   option lock-timer 180
#   option transport.address-family inet6
#   option base-port 49152
    option max-port  60999
end-volume

Any thoughts & suggestions very appreciated.

Many thanks, L.

________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/118564314

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/118564314

Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users


--
Thanks,
Sanju

okey, @devel - this might be worth more investigation & should be easy to reproduce.

These errors will show up if a user, in my case it was user's home fuse-mounted, is accessing some files, in my case it was just a shell session but(& maybe only in this specific scenario) with 'screen' program upon login - and (in my case) automount was restart (and possibly gluster's vol or gluster itself was re/started).

I spotted it here:

$ sudo journalctl -lf -o cat -u autofs
...
setautomntent: lookup(sss): setautomntent: No such file or directory
mounted indirect on /misc with timeout 300, freq 75 seconds
ghosting enabled
mounted indirect on /net with timeout 300, freq 75 seconds
ghosting enabled
mounted indirect on /0-ALL.DATA with timeout 300, freq 75 seconds
ghosting enabled
mounted indirect on /0-ALL.SYSDATA with timeout 300, freq 75 seconds
ghosting enabled
mounted indirect on /home with timeout 300, freq 75 seconds
re-connected to /home/jt455
ghosting enabled

The user got logged out(or any user accessing fuse-mounted gluster vol) and autofs restarted (errors gone) and gluster stopped spitting out:

0-glusterfs: connection attempt on 127.0.0.1:24007 failed, (Invalid argument)

many thanks, L

Attachment: pEpkey.asc
Description: application/pgp-keys

________

Community Meeting Calendar:

APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: https://bluejeans.com/118564314

NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: https://bluejeans.com/118564314

Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users

[Index of Archives]     [Gluster Development]     [Linux Filesytems Development]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux