Hi Soumya,
(1) In http://mseas.mit.edu/download/phaley/GlusterUsers/TestNFSmount/
I've placed the following 2 log files
etc-glusterfs-glusterd.vol.log
gdata.log
The first has repeated messages about nfs disconnects. The second had
the <fuse_mnt_direcotry>.log name (but not much information).
(2) About the gluster-NFS native server: do you know where we can find
documentation on how to use/install it? We haven't had success in our
searches.
Thanks
Pat
On 07/04/2017 05:01 AM, Soumya Koduri wrote:
On 07/03/2017 09:01 PM, Pat Haley wrote:
Hi Soumya,
When I originally did the tests I ran tcpdump on the client.
I have rerun the tests, doing tcpdump on the server
tcpdump -i any -nnSs 0 host 172.16.1.121 -w /root/capture_nfsfail.pcap
The results are in the same place
http://mseas.mit.edu/download/phaley/GlusterUsers/TestNFSmount/
capture_nfsfail.pcap has the results from the failed touch experiment
capture_nfssucceed.pcap has the results from the successful touch
experiment
The brick log files are there too.
Thanks for sharing. Looks like the error is not generated
@gluster-server side. The permission denied error was caused by either
kNFS or by fuse-mnt process or probably by the combination.
To check fuse-mnt logs, please look at
/var/log/glusterfs/<fuse_mnt_direcotry>.log
For eg.: if you have fuse mounted the gluster volume at /mnt/fuse-mnt
and exported it via kNFS, the log location for that fuse_mnt shall be
at /var/log/glusterfs/mnt-fuse-mnt.log
Also why not switch to either gluster-NFS native server or NFS-Ganesha
instead of using kNFS, as they are recommended NFS servers to use with
gluster?
Thanks,
Soumya
I believe we are using kernel-NFS exporting a fuse mounted gluster
volume. I am having Steve confirm this. I tried to find the fuse-mnt
logs but failed. Where should I look for them?
Thanks
Pat
On 07/03/2017 07:58 AM, Soumya Koduri wrote:
On 06/30/2017 07:56 PM, Pat Haley wrote:
Hi,
I was wondering if there were any additional test we could perform to
help debug the group write-permissions issue?
Sorry for the delay. Please find response inline --
Thanks
Pat
On 06/27/2017 12:29 PM, Pat Haley wrote:
Hi Soumya,
One example, we have a common working directory dri_fleat in the
gluster volume
drwxrwsr-x 22 root dri_fleat 4.0K May 1 15:14 dri_fleat
my user (phaley) does not own that directory but is a member of the
group dri_fleat and should have write permissions. When I go to the
nfs-mounted version and try to use the touch command I get the
following
ibfdr-compute-0-4(dri_fleat)% touch dum
touch: cannot touch `dum': Permission denied
One of the sub-directories under dri_fleat is "test" which phaley
owns
drwxrwsr-x 2 phaley dri_fleat 4.0K May 1 15:16 test
Under this directory (mounted via nfs) user phaley can write
ibfdr-compute-0-4(test)% touch dum
ibfdr-compute-0-4(test)%
I have put the packet captures in
http://mseas.mit.edu/download/phaley/GlusterUsers/TestNFSmount/
capture_nfsfail.pcap has the results from the failed touch
experiment
capture_nfssucceed.pcap has the results from the successful touch
experiment
The command I used for these was
tcpdump -i ib0 -nnSs 0 host 172.16.1.119 -w
/root/capture_nfstest.pcap
I hope these pkts were captured on the node where NFS server is
running. Could you please use '-i any' as I do not see glusterfs
traffic in the tcpdump.
Also looks like NFS v4 is used between client & nfs server. Are you
using kernel-NFS here (i.e, kernel-NFS exporting fuse mounted gluster
volume)?
If that is the case please capture fuse-mnt logs as well. This error
may well be coming from kernel-NFS itself before the request is sent
to fuse-mnt process.
FWIW, we have below option -
Option: server.manage-gids
Default Value: off
Description: Resolve groups on the server-side.
I haven't looked into what this option exactly does. But it may worth
testing with this option on.
Thanks,
Soumya
The brick log files are also in the above link. If I read them
correctly they both funny times. Specifically I see entries from
around 2017-06-27 14:02:37.404865 even though the system time was
2017-06-27 12:00:00.
One final item, another reply to my post had a link for possible
problems that could arise from users belonging to too many group. We
have seen the above problem even with a user belonging to only 4
groups.
Let me know what additional information I can provide.
Thanks
Pat
On 06/27/2017 02:45 AM, Soumya Koduri wrote:
On 06/27/2017 10:17 AM, Pranith Kumar Karampuri wrote:
The only problem with using gluster mounted via NFS is that it
does not
respect the group write permissions which we need.
We have an exercise coming up in the a couple of weeks. It seems
to me
that in order to improve our write times before then, it would be
good
to solve the group write permissions for gluster mounted via NFS
now.
We can then revisit gluster mounted via FUSE afterwards.
What information would you need to help us force gluster mounted
via
NFS
to respect the group write permissions?
Is this owning group or one of the auxiliary groups whose write
permissions are not considered? AFAIK, there are no special
permission checks done by gNFS server when compared to gluster
native
client.
Could you please provide simple steps to reproduce the issue and
collect pkt trace and nfs/brick logs as well.
Thanks,
Soumya
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Pat Haley Email: phaley@xxxxxxx
Center for Ocean Engineering Phone: (617) 253-6824
Dept. of Mechanical Engineering Fax: (617) 253-8125
MIT, Room 5-213 http://web.mit.edu/phaley/www/
77 Massachusetts Avenue
Cambridge, MA 02139-4301
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-users
