Re: [Gluster-devel] v3.6.2

"David F. Robinson" <david.robinson@xxxxxxxxxxxxx> · Tue, 27 Jan 2015 14:18:26 +0000

Turning off nfslock did not help.  Also, still getting these messages every 3-seconds:

[2015-01-27 14:16:12.921880] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument)
[2015-01-27 14:16:15.922431] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument)
[2015-01-27 14:16:18.923080] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument)
[2015-01-27 14:16:21.923748] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument)
[2015-01-27 14:16:24.924472] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument)
[2015-01-27 14:16:27.925192] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument)
[2015-01-27 14:16:30.925895] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument)
[2015-01-27 14:16:33.926563] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument)
[2015-01-27 14:16:36.927248] W [socket.c:611:__socket_rwv] 0-management: readv on /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid argument)

------ Original Message ------
From: "Kaushal M" <kshlmster@xxxxxxxxx>
To: "David F. Robinson" <david.robinson@xxxxxxxxxxxxx>
Cc: "Joe Julian" <joe@xxxxxxxxxxxxxxxx>; "Gluster Users" <gluster-users@xxxxxxxxxxx>; "Gluster Devel" <gluster-devel@xxxxxxxxxxx>
Sent: 1/27/2015 1:49:56 AM
Subject: Re: Re[2]: [Gluster-devel]  v3.6.2

Your nfs.log file has the following lines, 
```

[2015-01-26 20:06:58.298078] E [rpcsvc.c:1303:rpcsvc_program_register_portmap] 0-rpc-service: Could not register with portmap 100021 4 38468
[2015-01-26 20:06:58.298108] E [nfs.c:331:nfs_init_versions] 0-nfs: Program  NLM4 registration failed
```

The Gluster NFS server has it's own NLM (nlockmgr) implementation. You said that you have the nfslock service on. Can you turn that off and try again?

~kaushal

On Tue, Jan 27, 2015 at 11:21 AM, David F. Robinson <david.robinson@xxxxxxxxxxxxx> wrote:

On a different system where gluster-NFS (not kernel-nfs) is working properly shows the following:

[root@gfs01a glusterfs]# rpcinfo -p
   program vers proto   port  service
    100000    4   tcp    111  portmapper
    100000    3   tcp    111  portmapper
    100000    2   tcp    111  portmapper
    100000    4   udp    111  portmapper
    100000    3   udp    111  portmapper
    100000    2   udp    111  portmapper
    100005    3   tcp  38465  mountd
    100005    1   tcp  38466  mountd
    100003    3   tcp   2049  nfs
    100024    1   udp  42413  status
    100024    1   tcp  35424  status
    100021    4   tcp  38468  nlockmgr
    100021    1   udp    801  nlockmgr
    100227    3   tcp   2049  nfs_acl
    100021    1   tcp    804  nlockmgr

[root@gfs01a glusterfs]# /etc/init.d/nfs status
rpc.svcgssd is stopped
rpc.mountd is stopped
nfsd is stopped
rpc.rquotad is stopped

------ Original Message ------
From: "Joe Julian" <joe@xxxxxxxxxxxxxxxx>
To: "Kaushal M" <kshlmster@xxxxxxxxx>; "David F. Robinson" <david.robinson@xxxxxxxxxxxxx>
Cc: "Gluster Users" <gluster-users@xxxxxxxxxxx>; "Gluster Devel" <gluster-devel@xxxxxxxxxxx>

Sent: 1/27/2015 12:48:49 AM
Subject: Re: [Gluster-devel]  v3.6.2

If that was true, wouldn't it not "Connection refused" because the kernel nfs is listening? 

On January 26, 2015 9:43:34 PM PST, Kaushal M <kshlmster@xxxxxxxxx> wrote: 

Seems like you have the kernel NFS server running. The `rpcinfo -p` output you provided shows that there are other mountd, nfs and nlockmgr services running your system. Gluster NFS server requires that the kernel nfs services be disabled and not running. 

~kaushal

On Tue, Jan 27, 2015 at 10:56 AM, David F. Robinson <david.robinson@xxxxxxxxxxxxx> wrote:

[root@gfs01bkp ~]# gluster volume status homegfs_bkp
Status of volume: homegfs_bkp
Gluster process                                         Port    Online  Pid
------------------------------------------------------------------------------
Brick gfsib01bkp.corvidtec.com:/data/brick01bkp/homegfs
_bkp                                                    49152   Y       4087
Brick gfsib01bkp.corvidtec.com:/data/brick02bkp/homegfs
_bkp                                                    49155   Y       4092
NFS Server on localhost                                 N/A     N       N/A

Task Status of Volume homegfs_bkp
------------------------------------------------------------------------------
Task                 : Rebalance
ID                   : 6d4c6c4e-16da-48c9-9019-dccb7d2cfd66
Status               : completed 

------ Original Message ------
From: "Atin Mukherjee" <amukherj@xxxxxxxxxx>
To: "Pranith Kumar Karampuri" <pkarampu@xxxxxxxxxx>; "Justin Clift" <justin@xxxxxxxxxxx>; "David F. Robinson" <david.robinson@xxxxxxxxxxxxx>
Cc: "Gluster Users" <gluster-users@xxxxxxxxxxx>; "Gluster Devel" <gluster-devel@xxxxxxxxxxx>
Sent: 1/26/2015 11:51:13 PM
Subject: Re: [Gluster-devel] v3.6.2

On 01/27/2015 07:33 AM, Pranith Kumar Karampuri wrote:

 On 01/26/2015 09:41 PM, Justin Clift wrote:

 On 26 Jan 2015, at 14:50, David F. Robinson
 <david.robinson@xxxxxxxxxxxxx> wrote:

 I have a server with v3.6.2 from which I cannot mount using NFS. The
 FUSE mount works, however, I cannot get the NFS mount to work. From
 /var/log/message:
   Jan 26 09:27:28 gfs01bkp mount[2810]: mount to NFS server
 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying
 Jan 26 09:27:53 gfs01bkp mount[4456]: mount to NFS server
 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying
 Jan 26 09:29:28 gfs01bkp mount[2810]: mount to NFS server
 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying
 Jan 26 09:29:53 gfs01bkp mount[4456]: mount to NFS server
 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying
 Jan 26 09:31:28 gfs01bkp mount[2810]: mount to NFS server
 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying
 Jan 26 09:31:53 gfs01bkp mount[4456]: mount to NFS server
 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying
 Jan 26 09:33:28 gfs01bkp mount[2810]: mount to NFS server
 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying
 Jan 26 09:33:53 gfs01bkp mount[4456]: mount to NFS server
 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying
 Jan 26 09:35:28 gfs01bkp mount[2810]: mount to NFS server
 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying
 Jan 26 09:35:53 gfs01bkp mount[4456]: mount to NFS server
 'gfsib01bkp.corvidtec.com' failed: Connection refused, retrying
     I also am continually getting the following errors in
 /var/log/glusterfs:
   [root@gfs01bkp glusterfs]# tail -f etc-glusterfs-glusterd.vol.log
 [2015-01-26 14:41:51.260827] W [socket.c:611:__socket_rwv]
 0-management: readv on
 /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid
 argument)
 [2015-01-26 14:41:54.261240] W [socket.c:611:__socket_rwv]
 0-management: readv on
 /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid
 argument)
 [2015-01-26 14:41:57.261642] W [socket.c:611:__socket_rwv]
 0-management: readv on
 /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid
 argument)
 [2015-01-26 14:42:00.262073] W [socket.c:611:__socket_rwv]
 0-management: readv on
 /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid
 argument)
 [2015-01-26 14:42:03.262504] W [socket.c:611:__socket_rwv]
 0-management: readv on
 /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid
 argument)
 [2015-01-26 14:42:06.262935] W [socket.c:611:__socket_rwv]
 0-management: readv on
 /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid
 argument)
 [2015-01-26 14:42:09.263334] W [socket.c:611:__socket_rwv]
 0-management: readv on
 /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid
 argument)
 [2015-01-26 14:42:12.263761] W [socket.c:611:__socket_rwv]
 0-management: readv on
 /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid
 argument)
 [2015-01-26 14:42:15.264177] W [socket.c:611:__socket_rwv]
 0-management: readv on
 /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid
 argument)
 [2015-01-26 14:42:18.264623] W [socket.c:611:__socket_rwv]
 0-management: readv on
 /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid
 argument)
 [2015-01-26 14:42:21.265053] W [socket.c:611:__socket_rwv]
 0-management: readv on
 /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid
 argument)
 [2015-01-26 14:42:24.265504] W [socket.c:611:__socket_rwv]
 0-management: readv on
 /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket failed (Invalid
 argument)
 I believe this error message comes when the socket file is not present.
 I see the following commit which changed the location of the sockets.
 May be Atin may know more. about this: +Atin.

Can we get the output of 'gluster volume status' for the volume which
you are trying to mount?

~Atin

 Pranith

 ^C
   Also, when I try to NFS mount my gluster volume, I am getting
 Any chance there's a network or host based firewall stopping some of
 the ports?

 + Justin

 --
 GlusterFS - http://www.gluster.org/

 An open source, distributed file system scaling to several
 petabytes, and handling thousands of clients.

 My personal twitter: twitter.com/realjustinclift

 _______________________________________________
 Gluster-devel mailing list
 Gluster-devel@xxxxxxxxxxx
 http://www.gluster.org/mailman/listinfo/gluster-devel

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users

Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users