Have you tried CTDB from the Samba project I have user it with nfs-ganesha and samba4 before with great success, further more you do not need to enable samba to run it. Original Message From: Soumya Koduri Sent: Tuesday, March 22, 2016 09:18 To: ML Wong Cc: gluster-users@xxxxxxxxxxx Subject: Re: nfs-ganesha volume null errors On 03/21/2016 02:16 AM, ML Wong wrote: > Hello Soumya, > Thanks for answering my questions. > Question 1) I am still puzzled what VOL is still referring to. Is that a > variable/parameter that i can specify somewhere in the ganesha-ha.conf? > Any pointers will be very much appreciated. No it doesn't refer to any volume as it is a global option. The log message is misleading. > > 1) Those 3 test systems do not have firewalld running and SELinux > running. And i also verify corosync.conf is now empty. > # sestatus > SELinux status: disabled > > # firewall-cmd --zone=public --list-all > FirewallD is not running > > # ls -al /etc/corosync/corosync.conf > -rw-r--r-- 1 root root 0 Mar 20 12:54 /etc/corosync/corosync.conf > > 2) I also do not find pacemaker.log under /var/log, but i found the > following. Will these be the same: > # ls -al /var/log/pcsd/pcsd.log > -rw-r--r--. 1 root root 162322 Mar 20 13:26 /var/log/pcsd/pcsd.log > > In any case, that log is full of the following: > +++ > I, [2016-03-20T13:33:34.982311 #939] INFO -- : Running: > /usr/sbin/corosync-cmapctl totem.cluster_name > I, [2016-03-20T13:33:34.982459 #939] INFO -- : CIB USER: hacluster, > groups: > I, [2016-03-20T13:33:34.985984 #939] INFO -- : Return Value: 1 > +++ There should be pacemaker.log too.. Which version of pacemaker are you using? > > 3) /var/log/messages - it does not look ganesha passing the logs to this > file. But i see /var/log/ganesha.log - which i found out logging seem > to be sent to there from /etc/sysconfig/ganesha (OPTIONS="-L > /var/log/ganesha.log -f /etc/ganesha/ganesha.conf -N NIV_FULL_DEBUG"" > > After it failed to acquire the volume, the server will be filled with > the following in "ganesha.log", but the other 2 nodes in the cluster do > not have anything logged in ganesha.log. The other nodes have "E > [MSGID: 106062] [glusterd-op-sm.c:3728:glusterd_op_ac_unlock] > 0-management: Unable to acquire volname" logged in the > "etc-glusterfs-glusterd.vol.log" > +++ > 20/03/2016 13:37:32 : epoch 56ef059d : mlw-fusion1 : > ganesha.nfsd-5215[dbus_heartbeat] gsh_dbus_thread :DBUS :F_DBG :top of > poll loop > 20/03/2016 13:37:32 : epoch 56ef059d : mlw-fusion1 : > ganesha.nfsd-5215[dbus_heartbeat] gsh_dbus_thread :RW LOCK :F_DBG > :Acquired mutex 0x7fd38e3fe080 (&dbus_bcast_lock) at > /builddir/build/BUILD/nfs-ganesha-2.3.0/src/dbus/dbus_server.c:689 > 20/03/2016 13:37:32 : epoch 56ef059d : mlw-fusion1 : > ganesha.nfsd-5215[dbus_heartbeat] gsh_dbus_thread :RW LOCK :F_DBG > :Released mutex 0x7fd38e3fe080 (&dbus_bcast_lock) at > /builddir/build/BUILD/nfs-ganesha-2.3.0/src/dbus/dbus_server.c:739 > +++ > '/var/log/messages' is where cluster-setup related errors are logged. Probably to debug you could try below steps - * bring up nfs-ganesha server on all the nodes #systemctl start nfs-ganesha * Check if nfs-ganesha is successfully started * On one of the nodes, # cd '/usr/libexec/ganesha' # bash -x ./ganesha.sh --setup /etc/ganesha This would throw the errors returned by the script on the console during cluster setup. Please give a try and let me know if you see any errors. Thanks, Soumya > Testing Environment: Running CentOS Linux release 7.2.1511, glusterfs > 3.7.8 (glusterfs-server-3.7.8-2.el7.x86_64), > nfs-ganesha-gluster-2.3.0-1.el7.x86_64 > > > On Mon, Mar 14, 2016 at 2:05 AM, Soumya Koduri <skoduri@xxxxxxxxxx > <mailto:skoduri@xxxxxxxxxx>> wrote: > > Hi, > > > On 03/14/2016 04:06 AM, ML Wong wrote: > > Running CentOS Linux release 7.2.1511, glusterfs 3.7.8 > (glusterfs-server-3.7.8-2.el7.x86_64), > nfs-ganesha-gluster-2.3.0-1.el7.x86_64 > > 1) Ensured the connectivity between gluster nodes by using PING > 2) Disabled NetworkManager (Loaded: loaded > (/usr/lib/systemd/system/NetworkManager.service; disabled) > 3) Gluster 'gluster_shared_storage' is created by using (gluster > volume > set all cluster.enable-shared-storage enable), and are all > mounted under > /run/gluster/shared_storage, and nfs-ganesha directory is also > created > after the feature being enabled > 4) Emtpy out /etc/ganesha/ganesha.conf (have tested ganesha > running as a > stand-alone NFS server) > 5) Installed pacemaker, corosync, and resource-agents > 6) Reset 'hacluster' system-user password to be the same: > # pcs cluster auth -u hacluster mlw-fusion1 > mlw-fusion2 mlw-fusion3 > Password: > mlw-fusion2: Authorized > mlw-fusion3: Authorized > mlw-fusion1: Authorized > 7) IPv6 is enabled - (IPV6INIT=yes in > /etc/sysconfig/network-scripts/ifcfg-en*) > 8) Started pcsd, and corosync > 9) Created /var/lib/glusterd/nfs/secret.pem, and transfer to the > other 2 > nodes > # ssh -i secret.pem root@mlw-fusion3 "echo helloworld" > helloworld > 9) Transfer the following ganesha-ha.conf to the other nodes in the > cluster, but change the HA_VOL_SERVER value accordingly to > mlw-fusion2, > and mlw-fusion3 > > HA_NAME="ganesha-ha-01" > HA_VOL_SERVER="mlw-fusion1" > HA_CLUSTER_NODES="mlw-fusion1,mlw-fusion2,mlw-fusion3" > VIP_mlw_fusion1="192.168.30.201" > VIP_mlw_fusion2="192.168.30.202" > VIP_mlw_fusion3="192.168.30.203" > > Question 1) As i am new to nfs-ganesha, pacemaker, corosync, i was > mostly puzzled by the error message found in the > 'etc-glusterfs-glusterd.vol.log'. It seems like it will show the > below > message regardless of what i have done to troubleshoot - So, > what Volume > does these error messages are referring to? > > > Just a guess. Since this option is not tied to any particular > volume, it may have thrown (null) in the error message. Could you > check '/var/log/messages' and '/var/log/pacemaker.log' for the > errors/warnings. Since they are running RHEL 7, please check if > there are denials from selinux or firewalld. > > Is that referring to the > > HA_VOL_NAME in the /usr/libexec/ganesha/ganesha-ha.sh? Do I need to > change any of the 4 HA_* variables inside ganesha-ha.sh? > HA_NUM_SERVERS=0 > HA_SERVERS="" > HA_CONFDIR="/etc/ganesha" > HA_VOL_NAME="gluster_shared_storage" > HA_VOL_MNT="/run/gluster/shared_storage" > > No. You need not change any of these variables. > > > E [MSGID: 106123] [glusterd-syncop.c:1407:gd_commit_op_phase] > 0-management: Commit of operation 'Volume (null)' failed on > localhost : > Failed to set up HA config for NFS-Ganesha. Please check the log > file > for details > > Question 2) do I really have to start corosync before enabling > nfs-ganesha? > > No. The setup automatically starts pacemaker and corosync services. > > Thanks, > Soumya > > > Any help will be appreciated!!! > > > _______________________________________________ > Gluster-users mailing list > Gluster-users@xxxxxxxxxxx <mailto:Gluster-users@xxxxxxxxxxx> > http://www.gluster.org/mailman/listinfo/gluster-users > > _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-users _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-users
