On Thu, Nov 10, 2016 at 3:41 PM, Dan van der Ster <dan@xxxxxxxxxxxxxx> wrote: > Hi all, Hi Zheng, > > We're seeing a strange issue with the kernel cephfs clients, combined > with a path restricted mds cap. It seems that files/dirs are > intermittently not created due to permission denied. > > For example, when I untar a kernel into cephfs, we see ~1/1000 files > failed to open/mkdir. > Client caps are in the PS [1]. > We've tried kernels 3.10.0-493.el7, 4.8.6, and 4.9-rc4 -- all have the > same intermittent behaviour. We could *not* reproduce the issue with > ceph-fuse 10.2.3. > > The cluster is running 10.2.3. > > Now, if we remove the path restricted cap -- i.e. use mds 'allow rw' > -- then we have no more errors. > > So it seems there is a race in the path restriction cap code. > We grabbed an mds log, and noticed that it seems that a file is opened > twice, then the second open fails with 'already xlocked'. A full log > for one such file is here: http://pastebin.com/raw/YyULfjND > > Is anyone successfully using path caps with kernel clients? Maybe this > is a new bug? I don't have anything very intelligent to say about this other than that I've created a ticket for it here: http://tracker.ceph.com/issues/17893 We don't have any automated tests that cover this case at the moment, but it would be a good idea to create one. John > Cheers, Dan > > > [1] > [client.k8s] > key: xxx== > caps: [mds] allow r, allow rw path=/k8s > caps: [mon] allow r > caps: [osd] allow rw pool=cephfs_data_k8s > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |