Hi Goncalo, Thank you for those links. It appears that that fix was already in the 10.2.3 mds, which we are running. I've just upgraded the mds's to the current jewel gitbuilder (10.2.3-358.g427f357.x86_64) and the problem is still there. (BTW, in testing this I've been toggling the mds caps between 'allow rw' and 'allow r, allow rw path=/k8s'. During this I've noticed that the new caps are only applied after I remount.) Cheers, Dan On Fri, Nov 11, 2016 at 1:29 AM, Goncalo Borges <goncalo.borges@xxxxxxxxxxxxx> wrote: > Hi Dan,,, > > I know there are path restriction issues in the kernel client. See the discussion here. > > http://lists.opennebula.org/pipermail/ceph-users-ceph.com/2016-June/010656.html > > http://tracker.ceph.com/issues/16358 > > Cheers > Goncalo > > ________________________________________ > From: ceph-users [ceph-users-bounces@xxxxxxxxxxxxxx] on behalf of Dan van der Ster [dan@xxxxxxxxxxxxxx] > Sent: 11 November 2016 02:41 > To: ceph-users; Yan, Zheng > Subject: Intermittent permission denied using kernel client with mds path cap > > Hi all, Hi Zheng, > > We're seeing a strange issue with the kernel cephfs clients, combined > with a path restricted mds cap. It seems that files/dirs are > intermittently not created due to permission denied. > > For example, when I untar a kernel into cephfs, we see ~1/1000 files > failed to open/mkdir. > Client caps are in the PS [1]. > We've tried kernels 3.10.0-493.el7, 4.8.6, and 4.9-rc4 -- all have the > same intermittent behaviour. We could *not* reproduce the issue with > ceph-fuse 10.2.3. > > The cluster is running 10.2.3. > > Now, if we remove the path restricted cap -- i.e. use mds 'allow rw' > -- then we have no more errors. > > So it seems there is a race in the path restriction cap code. > We grabbed an mds log, and noticed that it seems that a file is opened > twice, then the second open fails with 'already xlocked'. A full log > for one such file is here: http://pastebin.com/raw/YyULfjND > > Is anyone successfully using path caps with kernel clients? Maybe this > is a new bug? > > Cheers, Dan > > > [1] > [client.k8s] > key: xxx== > caps: [mds] allow r, allow rw path=/k8s > caps: [mon] allow r > caps: [osd] allow rw pool=cephfs_data_k8s > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |