Re: Intermittent permission denied using kernel client with mds path cap

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Goncalo,

Thank you for those links. It appears that that fix was already in the
10.2.3 mds, which we are running. I've just upgraded the mds's to the
current jewel gitbuilder (10.2.3-358.g427f357.x86_64) and the problem
is still there.

(BTW, in testing this I've been toggling the mds caps between 'allow
rw' and 'allow r, allow rw path=/k8s'. During this I've noticed that
the new caps are only applied after I remount.)

Cheers, Dan


On Fri, Nov 11, 2016 at 1:29 AM, Goncalo Borges
<goncalo.borges@xxxxxxxxxxxxx> wrote:
> Hi Dan,,,
>
> I know there are path restriction issues in the kernel client. See the discussion here.
>
> http://lists.opennebula.org/pipermail/ceph-users-ceph.com/2016-June/010656.html
>
> http://tracker.ceph.com/issues/16358
>
> Cheers
> Goncalo
>
> ________________________________________
> From: ceph-users [ceph-users-bounces@xxxxxxxxxxxxxx] on behalf of Dan van der Ster [dan@xxxxxxxxxxxxxx]
> Sent: 11 November 2016 02:41
> To: ceph-users; Yan, Zheng
> Subject:  Intermittent permission denied using kernel client        with mds path cap
>
> Hi all, Hi Zheng,
>
> We're seeing a strange issue with the kernel cephfs clients, combined
> with a path restricted mds cap. It seems that files/dirs are
> intermittently not created due to permission denied.
>
> For example, when I untar a kernel into cephfs, we see ~1/1000 files
> failed to open/mkdir.
> Client caps are in the PS [1].
> We've tried kernels 3.10.0-493.el7, 4.8.6, and 4.9-rc4 -- all have the
> same intermittent behaviour. We could *not* reproduce the issue with
> ceph-fuse 10.2.3.
>
> The cluster is running 10.2.3.
>
> Now, if we remove the path restricted cap -- i.e. use mds 'allow rw'
> -- then we have no more errors.
>
> So it seems there is a race in the path restriction cap code.
> We grabbed an mds log, and noticed that it seems that a file is opened
> twice, then the second open fails with 'already xlocked'. A full log
> for one such file is here: http://pastebin.com/raw/YyULfjND
>
> Is anyone successfully using path caps with kernel clients? Maybe this
> is a new bug?
>
> Cheers, Dan
>
>
> [1]
> [client.k8s]
>    key: xxx==
>    caps: [mds] allow r, allow rw path=/k8s
>    caps: [mon] allow r
>    caps: [osd] allow rw pool=cephfs_data_k8s
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux