On Fri, Jun 12, 2015 at 03:17:08PM +0000, David Robinson wrote: > My NFS stopped working after upgrading to 3.6.3. When I do a gluster volume > status homegfs_bkp, it shows as N/A and I cannot mount the volume using NFS > instead of FUSE. > Any suggestions for how to fix? > > [root@gfs01bkp glusterfs]# gluster volume status homegfs_bkp > Status of volume: homegfs_bkp > Gluster process Port Online Pid > ------------------------------------------------------------------------------ > Brick gfsib01bkp.corvidtec.com:/data/brick01bkp/homegfs > _bkp 49152 Y 4311 > Brick gfsib01bkp.corvidtec.com:/data/brick02bkp/homegfs > _bkp 49155 Y 4318 > NFS Server on localhost N/A N N/A > > Task Status of Volume homegfs_bkp > ------------------------------------------------------------------------------ > Task : Rebalance > ID : 6d4c6c4e-16da-48c9-9019-dccb7d2cfd66 > Status : completed > > > From the /var/log/glusterfs/nfs.log file: > > st_atim.tv_nsec 1 > package-string: glusterfs 3.6.3 > [2015-06-12 15:05:09.184368] W [glusterfsd.c:1194:cleanup_and_exit] (--> 0-: > received signum (0), shutting down > [2015-06-12 15:09:40.589589] I [MSGID: 100030] [glusterfsd.c:2018:main] > 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.6.3 > (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/nfs -p > /var/lib/glusterd/nfs/run/nfs.pid -l /var/log/glusterfs/nfs.log -S > /var/run/1f0cee5a2d074e39b32ee5a81c70e68c.socket) > [2015-06-12 15:09:40.622811] I > [rpcsvc.c:2142:rpcsvc_set_outstanding_rpc_limit] 0-rpc-service: Configured > rpc.outstanding-rpc-limit with value 16 > [2015-06-12 15:09:40.683881] E > [rpcsvc.c:1303:rpcsvc_program_register_portmap] 0-rpc-service: Could not > register with portmap 100021 4 38468 > [2015-06-12 15:09:40.683907] E [nfs.c:331:nfs_init_versions] 0-nfs: Program > NLM4 registration failed > [2015-06-12 15:09:40.683926] E [nfs.c:1342:init] 0-nfs: Failed to initialize > protocols These messages suggest that the portmapper/rpcbind already has a NLM service registered. Gluster/NFS tries to be friendly to other services and does not kick them out. Sometimes it happens that there is a stale registration left behind... One cause was a change in the rpcbind systemd unit, see https://bugzilla.redhat.com/1181779 for more details. If you are on RHEL-7 or CentOS-7, this solution is known to work: # sed "s/ -w//" /usr/lib/systemd/system/rpcbind.service > \ /etc/systemd/system/rpcbind.service # systemctl daemon-reload # systemctl restart rpcbind # systemctl restart glusterd Alternatively you can check with 'rpcinfo -p' which ports are registered at the portmapper, and check with 'netstat' or 'ss' if there is a process listening on these ports. You can manually unregister the (stale) ports with 'rpcinfo -d 100021 4' (NLM, version 4). HTH, Niels > [2015-06-12 15:09:40.683933] E [xlator.c:425:xlator_init] 0-nfs-server: > Initialization of volume 'nfs-server' failed, review your volfile again > [2015-06-12 15:09:40.683939] E [graph.c:322:glusterfs_graph_init] > 0-nfs-server: initializing translator failed > [2015-06-12 15:09:40.683945] E [graph.c:525:glusterfs_graph_activate] > 0-graph: init failed > pending frames: > frame : type(0) op(0) > patchset: git://git.gluster.com/glusterfs.git > signal received: 11 > time of crash: > 2015-06-12 15:09:40 > configuration details: > argp 1 > backtrace 1 > dlfcn 1 > libpthread 1 > llistxattr 1 > setfsid 1 > spinlock 1 > epoll.h 1 > xattr.h 1 > st_atim.tv_nsec 1 > package-string: glusterfs 3.6.3 > > Gluster version: > [root@gfs01bkp glusterfs]# rpm -qa | grep gluster > glusterfs-server-3.6.3-1.el6.x86_64 > glusterfs-3.6.3-1.el6.x86_64 > glusterfs-rdma-3.6.3-1.el6.x86_64 > glusterfs-api-3.6.3-1.el6.x86_64 > glusterfs-devel-3.6.3-1.el6.x86_64 > glusterfs-api-devel-3.6.3-1.el6.x86_64 > glusterfs-libs-3.6.3-1.el6.x86_64 > glusterfs-extra-xlators-3.6.3-1.el6.x86_64 > glusterfs-geo-replication-3.6.3-1.el6.x86_64 > samba-vfs-glusterfs-4.1.11-2.el6.x86_64 > glusterfs-fuse-3.6.3-1.el6.x86_64 > glusterfs-debuginfo-3.6.3-1.el6.x86_64 > glusterfs-cli-3.6.3-1.el6.x86_64 > > > [root@gfs01bkp glusterfs]# gluster volume info homegfs_bkp > > Volume Name: homegfs_bkp > Type: Distribute > Volume ID: 96de8872-d957-4205-bf5a-076e3f35b294 > Status: Started > Number of Bricks: 2 > Transport-type: tcp > Bricks: > Brick1: gfsib01bkp.corvidtec.com:/data/brick01bkp/homegfs_bkp > Brick2: gfsib01bkp.corvidtec.com:/data/brick02bkp/homegfs_bkp > > > ======================== > > > > David F. Robinson, Ph.D. > > President - Corvid Technologies > > 145 Overhill Drive > > Mooresville, NC 28117 > > 704.799.6944 x101 [Office] > > 704.252.1310 [Cell] > > 704.799.7974 [Fax] > > david.robinson@xxxxxxxxxxxxx > > http://www.corvidtec.com > > _______________________________________________ > Gluster-devel mailing list > Gluster-devel@xxxxxxxxxxx > http://www.gluster.org/mailman/listinfo/gluster-devel _______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-devel