Fwd: gluster working, but error appearing every two seconds in logs - NEW INFO

[Basavanagowda Kanur <gowda@xxxxxxxxxxxxx>]

Forwarding to list

---------- Forwarded message ----------
From: Basavanagowda Kanur <gowda@xxxxxxxxxxx>
Date: Thu, Feb 19, 2009 at 11:56 PM
Subject: Re: gluster working, but error appearing every two seconds in logs - NEW INFO
To: jordi@xxxxxxxxx
Cc: Anand Avati <avati@xxxxxxxxxxxxx>, Gluster List <gluster-devel@xxxxxxxxxx>

Jordi,
Please find replies inline.

Hi,

since the last time we were in contact I've been trying to track down where the problem is. I've been monitoring almost every possible thing related to network traffic, and... eventually.... i found out what the problem is by chance!!

Did you try 'telnet <server> <port>' at the time when you were getting 'Connection refused' errors?

It turns out that when in a client-server mounting gluster i run "df -h", i get this:

***********
2009-02-19 17:15:43 E [tcp-client.c:190:tcp_connect] espai1: non-blocking connect() returned: 111 (Connection refused)
2009-02-19 17:15:43 W [client-protocol.c:332:client_protocol_xfer] espai1: not connected at the moment to submit frame type(1) op(15)
2009-02-19 17:15:43 E [client-protocol.c:3891:client_statfs_cbk] espai1: no proper reply from server, returning ENOTCONN
2009-02-19 17:15:43 E [tcp-client.c:190:tcp_connect] espai5: non-blocking connect() returned: 111 (Connection refused)
2009-02-19 17:15:43 W [client-protocol.c:332:client_protocol_xfer] espai5: not connected at the moment to submit frame type(1) op(15)
2009-02-19 17:15:43 E [client-protocol.c:3891:client_statfs_cbk] espai5: no proper reply from server, returning ENOTCONN
2009-02-19 17:15:43 E [tcp-client.c:190:tcp_connect] espai2: non-blocking connect() returned: 111 (Connection refused)
2009-02-19 17:15:43 W [client-protocol.c:332:client_protocol_xfer] espai2: not connected at the moment to submit frame type(1) op(15)
2009-02-19 17:15:43 E [client-protocol.c:3891:client_statfs_cbk] espai2: no proper reply from server, returning ENOTCONN
2009-02-19 17:15:43 E [tcp-client.c:190:tcp_connect] espai6: non-blocking connect() returned: 111 (Connection refused)
2009-02-19 17:15:43 W [client-protocol.c:332:client_protocol_xfer] espai6: not connected at the moment to submit frame type(1) op(15)
2009-02-19 17:15:43 E [client-protocol.c:3891:client_statfs_cbk] espai6: no proper reply from server, returning ENOTCONN

************

so... the reason why it is appearing so often is that i've got munin monitoring this gluster environment, and it performs a "df" command to check the disk space of all the servers, including, of course, the gluster mount point. When this happens... the error log shown above these lines is reported and eventually.... the mount point in that server fails. No data is lost, but i have to remount glusterfs as it becomes stale and data is not accessible.

is this a normal behaviour?

No. This is not normal behaviour. 'Connection refused' error comes when either the server is not running or there is a packet filtering between server and client.
 

i could stop munin from running "df" every 5 minutes... but still... is there any problem in my setup or is this what gluster is supposed to do?

The 'Connection refused' problem is not caused by any application accessing the mount-point. 'Connection refused' can come only if server is dead or packet is being filtered out.

Thanks.


_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxx
http://lists.nongnu.org/mailman/listinfo/gluster-devel

--
gowda

-- 
hard work often pays off after time, but laziness always pays off now