Hi Ben,
Current NFS-Ganesha version (>V2.2.0) support pNFS with Gluster but not
with HA. That means you need to disable HA to use pNFS. More details are
in the doc -
https://github.com/gluster/glusterfs/blob/master/doc/features/mount_gluster_volume_using_pnfs.md
For HA, as mentioned in the demo video, there are certain pre-requisites
to be followed. Noting down them below -
1. Populate /etc/hosts (or ensure DNS) on all machines.
2. Disable and stop NetworkManager service, enable and start network
service on all machines
3. Create a gluster shared volume 'gluster_shared_storage' and mount it
on '/var/run/gluster/shared_storage' on all the cluster nodes using
glusterfs native mount.
4. Touch /etc/ganesha/ganesha.conf (empty file)
5.. `yum -y install pacemaker cman pcs ccs resource-agents corosync` on
all machines
6. set cluster auth password on all machines
- `echo hacluster | passwd --stdin hacluster`
- pcs cluster auth on all the nodes
7. Enable IPv6 on all the cluster nodes.
8. Enable and start pcsd on all machines
- `chkconfig --add pcsd; chkconfig pcsd on; service pcsd start`
9. Populate /etc/ganesha/ganesha-ha.conf on all the cluster nodes.
Please make sure if you have followed all of them. From the logs, looks
like step3 and step4 are missed.
Thanks,
Soumya
On 06/03/2015 03:29 PM, 莊尚豪 wrote:
Sorry I post wrong command.
I want to turn on ganesha feature like your command.
And these are the output.
[root@n1 ~]# gluster nfs-ganesha enable
Enabling NFS-Ganesha requires Gluster-NFS to be disabled across the trusted pool. Do you still want to continue? (y/n) y
nfs-ganesha: failed: Commit failed on localhost. Please check the log file for more details.
And these are the detail log about turn on ganesha feature
[2015-06-03 09:56:19.951124] W [common-utils.c:1676:gf_string2boolean] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x147)[0x7f9937dc0b77] (--> /lib64/libglusterfs.so.0(gf_string2boolean+0x13f)[0x7f9937dd8ccf] (--> /usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_op_stage_set_ganesha+0xaa)[0x7f992cc0a32a] (--> /usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_op_stage_validate+0x9b)[0x7f992cbbf76b] (--> /usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(gd_stage_op_phase+0x1bd)[0x7f992cc369ed] ))))) 0-management: argument invalid
[2015-06-03 09:56:19.951312] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped
[2015-06-03 09:56:19.952278] I [glusterd-ganesha.c:364:check_host_list] 0-management: couldn't open the file /etc/ganesha/ganesha-ha.conf
[2015-06-03 09:56:19.957777] E [glusterd-ganesha.c:670:pre_setup] 0-THIS->name: mkdir() failed on path /var/run/gluster/shared_storage/nfs-ganesha,errno: 沒有此一檔案或目錄
[2015-06-03 09:56:19.957874] E [glusterd-ganesha.c:254:glusterd_op_set_ganesha] 0-management: Initial NFS-Ganesha set up failed
[2015-06-03 09:56:19.957899] E [glusterd-syncop.c:1396:gd_commit_op_phase] 0-management: Commit of operation 'Volume (null)' failed on localhost
By the way, does glusterfs HA for ganesha support pnfs?
Many thanks,
Ben
-----Original Message-----
From: Soumya Koduri [mailto:skoduri@xxxxxxxxxx]
Sent: Wednesday, June 03, 2015 3:35 PM
To: 莊尚豪; 'Anoop C S'; gluster-users@xxxxxxxxxxx
Cc: 'Meghana Madhusudhan'
Subject: Re: gluster-3.7 cannot start volume & ganesha feature cannot turn on problem
On 06/03/2015 12:01 PM, 莊尚豪 wrote:
Thank for your reply.
I have already upgrade glusterfs 3.7.1 and I can start the volume!
Therefore I am going to turn on ganesha feature by glusterfs.
[root@n1 yum.repos.d]# gluster volume set ganesha ganesha.enable on
volume set: failed: The option nfs-ganesha should be enabled before setting ganesha.enable.
This error means that 'nfs-ganesha' feature is not enabled. It needs to be enabled prior to exporting any volumes via ganesha.
cmd to be used - 'gluster nfs-ganesha enable'.
In case if you already have it enabled, check if it was successful using below commands -
'showmount -e localhost' --> should display '/'
'pcs status' --> Should display all the nodes' (which you have configured for HA ) pcs status with out any errors reported.
Thanks,
Soumya
There is the log in "/var/log/glusterfs/etc-glusterfs-glusterd.vol.log"
[2015-06-03 06:12:29.976469] W [common-utils.c:1676:gf_string2boolean]
(--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x147)[0x7f9937dc0b77]
(--> /lib64/libglusterfs.so.0(gf_string2boolean+0x13f)[0x7f9937dd8ccf]
(-->
/usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_op_stage_s
et_ganesha+0xaa)[0x7f992cc0a32a] (-->
/usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(glusterd_op_stage_v
alidate+0x9b)[0x7f992cbbf76b] (-->
/usr/lib64/glusterfs/3.7.1/xlator/mgmt/glusterd.so(gd_stage_op_phase+0
x1bd)[0x7f992cc369ed] ))))) 0-management: argument invalid
[2015-06-03 06:12:29.976823] I [MSGID: 106132]
[glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already
stopped
[2015-06-03 06:12:29.976938] I
[glusterd-ganesha.c:364:check_host_list] 0-management: couldn't open
the file /etc/ganesha/ganesha-ha.conf
[2015-06-03 06:12:29.984693] E [glusterd-ganesha.c:670:pre_setup]
0-THIS->name: mkdir() failed on path
/var/run/gluster/shared_storage/nfs-ganesha,errno: 沒有此一檔案或目錄
[2015-06-03 06:12:29.984851] E
[glusterd-ganesha.c:254:glusterd_op_set_ganesha] 0-management: Initial
NFS-Ganesha set up failed
[2015-06-03 06:12:29.984883] E
[glusterd-syncop.c:1396:gd_commit_op_phase] 0-management: Commit of
operation 'Volume (null)' failed on localhost
The Chinese log means that the path is empty or no directory.
It seems that glusterfs does not build ganesha configuration in initialization.
How do I continue next step?
Thanks,
Ben
-----Original Message-----
From: Anoop C S [mailto:achiraya@xxxxxxxxxx]
Sent: Tuesday, June 02, 2015 11:18 PM
To: benchuang@xxxxxxxxxx; gluster-users@xxxxxxxxxxx
Cc: Meghana Madhusudhan; Soumya Koduri
Subject: Re: gluster-3.7 cannot start volume & ganesha
feature cannot turn on problem
Can you please attach the glusterd logs here? You are having trouble to even start the volume here right?
And also HA configuration is mandatory to use NFS-Ganesha in this release. Once you have the volume started, I can help you with the remaining steps in detail.
Thanks
Meghana
----- Original Message -----
From: "Anoop C S" <achiraya@xxxxxxxxxx>
To: gluster-users@xxxxxxxxxxx
Cc: "Meghana Madhusudhan" <mmadhusu@xxxxxxxxxx>, "Soumya Koduri"
<skoduri@xxxxxxxxxx>
Sent: Tuesday, June 2, 2015 4:38:39 PM
Subject: Re: gluster-3.7 cannot start volume & ganesha
feature cannot turn on problem manadatroy
On 06/02/2015 01:42 PM, 莊尚豪 wrote:
Hi all,
I have two question for glusterfs-3.7 on fedora-22
I used to have a glusterfs cluster version 3.6.2.
The following configuration can be work in version-3.6.2, but not in
version-3.7
There is 2 node for glusterfs.
OS: fedora 22
Gluster: 3.7 on
https://download.gluster.org/pub/gluster/glusterfs/3.7/3.7.0/
#gluster peer probe n1
#gluster volume create ganesha n1:/data/brick1/gv0
n2:/data/brick1/gv0
Volume Name: ganesha
Type: Distribute
Volume ID: cbb8d360-0025-419c-a12b-b29e4b91d7f8
Status: Created
Number of Bricks: 2
Transport-type: tcp
Bricks:
Brick1: n1:/data/brick1/gv0
Brick2: n2:/data/brick1/gv0
Options Reconfigured:
performance.readdir-ahead: on
The problem to start the volume "ganesha"
#gluster volume start ganesha
volume start: ganesha: failed: Commit failed on localhost. Please
check the log file for more details.
LOG in /var/log/glusterfs/bricks/data-brick1-gv0.log
[2015-06-02 08:02:55.232923] I [MSGID: 100030]
[glusterfsd.c:2294:main] 0-/usr/sbin/glusterfsd: Started running
/usr/sbin/glusterfsd version 3.7.0 (args: /usr/sbin/glusterfsd -s
n2 --volfile-id ganesha.n2.data-brick1-gv0 -p
/var/lib/glusterd/vols/ganesha/run/n2-data-brick1-gv0.pid -S
/var/run/gluster/73ea8a39514304f5ebd440321d784386.socket
--brick-name /data/brick1/gv0 -l
/var/log/glusterfs/bricks/data-brick1-gv0.log --xlator-option
*-posix.glusterd-uuid=35547067-d343-4fee-802a-0e911b5a07cd
--brick-port 49157 --xlator-option
ganesha-server.listen-port=49157)
[2015-06-02 08:02:55.284206] I
[event-epoll.c:629:event_dispatch_epoll_worker] 0-epoll: Started
thread with index 1
[2015-06-02 08:02:55.397923] W [xlator.c:192:xlator_dynload]
0-xlator: /usr/lib64/glusterfs/3.7.0/xlator/features/changelog.so:
undefined symbol: changelog_select_event
This particular error for "undefined symbol changelog_select_event"
was identified recently and corresponding fix [
http://review.gluster.org/#/c/11004/ ] is already in master and
hopefully will be available with v3.7.1.
[2015-06-02 08:02:55.397963] E [graph.y:212:volume_type] 0-parser:
Volume 'ganesha-changelog', line 30: type 'features/changelog' is not
valid or not found on this machine
[2015-06-02 08:02:55.397992] E [graph.y:321:volume_end] 0-parser:
"type" not specified for volume ganesha-changelog
[2015-06-02 08:02:55.398214] E [MSGID: 100026]
[glusterfsd.c:2149:glusterfs_process_volfp] 0-: failed to construct
the graph
[2015-06-02 08:02:55.398423] W
[glusterfsd.c:1219:cleanup_and_exit] (--> 0-: received signum (0),
shutting down
I cannot google method to resolve it.
Does anyone have across this problem?
Another question is the feature in nfs-ganesha(version 2.2)
The volume command I cannot turn on this feature.
I try to copy the demo glusterfs-ganesha video but cannot work.
Demo link:
https://plus.google.com/events/c9omal6366f2cfkcd0iuee5ta1o
[root@n1 brick1]# gluster nfs-ganesha enable
Enabling NFS-Ganesha requires Gluster-NFS to be disabled across the
trusted pool. Do you still want to continue? (y/n) y
nfs-ganesha: failed: Commit failed on localhost. Please check the log
file for more details.
Adding ganesha folks to the thread.
Does anyone have the detail configuration?
THANKS for giving advice.
Regards,
Ben
_______________________________________________ Gluster-users mailing
list Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users
