Re: Slow write times to gluster disk

Steve Postma <SPostma@xxxxxxxxxxxx> · Mon, 7 Aug 2017 16:05:09 +0000



    Hi Soumya,

    
    We just had the opportunity to try the option of disabling the
    kernel-NFS and restarting glusterd to start gNFS.  However the
    gluster demon crashes immediately on startup.  What additional
    information besides what we provide below would help debugging this?

    
    Thanks,

    
    Pat

    
      -------- Forwarded Message --------
      
        
            Subject:
            
            gluster-nfs crashing on start
          
          
            Date: 
            Mon, 7 Aug 2017 16:05:09 +0000
          
          
            From: 
            Steve Postma <SPostma@xxxxxxxxxxxx>
          
          
            To: 
            Pat Haley <phaley@xxxxxxx>
          
        
          To
              disable kernal-nfs and enable nfs through Gluster we:
          

          gluster
            volume set data-volume nfs.export-volumes on

            gluster volume set data-volume nfs.disable off
          /etc/init.d/glusterd
              stop
          

              service nfslock stop
            
              service rpcgssd stop
            
              service rpcidmapd stop
            
              service portmap stop
            
              service nfs stop
          
          
           /etc/init.d/glusterd
              stop
          

          the
              /var/log/glusterfs/nfs.log immediately
              reports a crash:
          

            [root@mseas-data2
              glusterfs]# cat nfs.log
            [2017-08-07
              15:20:16.327026] I [MSGID: 100030]
              [glusterfsd.c:2332:main] 0-/usr/sbin/glusterfs: Started
              running /usr/sbin/glusterfs version 3.7.11 (args:
              /usr/sbin/glusterfs -s localhost --volfile-id gluster/nfs
              -p /var/lib/glusterd/nfs/run/nfs.pid -l
              /var/log/glusterfs/nfs.log -S
              /var/run/gluster/7db74f19472511d20849e471bf224c1a.socket)
            [2017-08-07
              15:20:16.345166] I [MSGID: 101190]
              [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll:
              Started thread with index 1
            [2017-08-07
              15:20:16.351290] I
              [rpcsvc.c:2215:rpcsvc_set_outstanding_rpc_limit]
              0-rpc-service: Configured rpc.outstanding-rpc-limit with
              value 16
            pending
              frames:
            frame
              : type(0) op(0)
            patchset:
              git://git.gluster.com/glusterfs.git
            signal
              received: 11
            time
              of crash:
            2017-08-07
              15:20:17
            configuration
              details:
            argp
              1
            backtrace
              1
            dlfcn
              1
            libpthread
              1
            llistxattr
              1
            setfsid
              1
            spinlock
              1
            epoll.h
              1
            xattr.h
              1
            st_atim.tv_nsec
              1
            package-string:
              glusterfs 3.7.11
            /usr/lib64/libglusterfs.so.0(_gf_msg_backtrace_nomem+0xb8)[0x3889625a18]
            /usr/lib64/libglusterfs.so.0(gf_print_trace+0x32f)[0x38896456af]
            /lib64/libc.so.6[0x34a1c32660]
            /lib64/libc.so.6[0x34a1d3382f]
            /usr/lib64/glusterfs/3.7.11/xlator/nfs/server.so(+0x53307)[0x7f8d071b3307]
            /usr/lib64/glusterfs/3.7.11/xlator/nfs/server.so(exp_file_parse+0x302)[0x7f8d071b3742]
            /usr/lib64/glusterfs/3.7.11/xlator/nfs/server.so(mnt3_auth_set_exports_auth+0x45)[0x7f8d071b47a5]
            /usr/lib64/glusterfs/3.7.11/xlator/nfs/server.so(_mnt3_init_auth_params+0x91)[0x7f8d07183e41]
            /usr/lib64/glusterfs/3.7.11/xlator/nfs/server.so(mnt3svc_init+0x218)[0x7f8d07184228]
            /usr/lib64/glusterfs/3.7.11/xlator/nfs/server.so(nfs_init_versions+0xd7)[0x7f8d07174a37]
            /usr/lib64/glusterfs/3.7.11/xlator/nfs/server.so(init+0x77)[0x7f8d071767c7]
            /usr/lib64/libglusterfs.so.0(xlator_init+0x52)[0x3889622a82]
            /usr/lib64/libglusterfs.so.0(glusterfs_graph_init+0x31)[0x3889669aa1]
            /usr/lib64/libglusterfs.so.0(glusterfs_graph_activate+0x57)[0x3889669bd7]
            /usr/sbin/glusterfs(glusterfs_process_volfp+0xed)[0x405c0d]
            /usr/sbin/glusterfs(mgmt_getspec_cbk+0x312)[0x40dbd2]
            /usr/lib64/libgfrpc.so.0(rpc_clnt_handle_reply+0xa5)[0x3889e0f7b5]
            /usr/lib64/libgfrpc.so.0(rpc_clnt_notify+0x1a1)[0x3889e10891]
            /usr/lib64/libgfrpc.so.0(rpc_transport_notify+0x28)[0x3889e0bbd8]
            /usr/lib64/glusterfs/3.7.11/rpc-transport/socket.so(+0x94cd)[0x7f8d088e04cd]
            /usr/lib64/glusterfs/3.7.11/rpc-transport/socket.so(+0xa79d)[0x7f8d088e179d]
            /usr/lib64/libglusterfs.so.0[0x388968b2f0]
            /lib64/libpthread.so.0[0x34a2007aa1]
            /lib64/libc.so.6(clone+0x6d)[0x34a1ce8aad]
          
          
            [root@mseas-data2
              glusterfs]# gluster volume info
             
            Volume
              Name: data-volume
            Type:
              Distribute
            Volume
              ID: c162161e-2a2d-4dac-b015-f31fd89ceb18
            Status:
              Started
            Number
              of Bricks: 2
            Transport-type:
              tcp
            Bricks:
            Brick1:
              mseas-data2:/mnt/brick1
            Brick2:
              mseas-data2:/mnt/brick2
            Options
              Reconfigured:
            nfs.export-volumes:
              on
            nfs.disable:
              off
            performance.readdir-ahead:
              on
            diagnostics.brick-sys-log-level:
              WARNING
            nfs.exports-auth-enable:
              on
          
          
            "/var/lib/glusterd/nfs/exports"
            /gdata
              172.16.1.0/255.255.255.0(rw)
            

            What
                else can we do to identify why this is failing?

              
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-users