I filled http://tracker.ceph.com/issues/17858 recently, I am seeing this
problem on 10.2.3 ceph-fuse, but maybe kernel client is affected too.
It is easy to replicate, just do deep "mkdir -p", e.g. "mkdir -p
1/2/3/4/5/6/7/8/9/0/1/2/3/4/5/6/7/8/9"
On 16-11-11 10:46, Dan van der Ster wrote:
Hi Goncalo,
Thank you for those links. It appears that that fix was already in the
10.2.3 mds, which we are running. I've just upgraded the mds's to the
current jewel gitbuilder (10.2.3-358.g427f357.x86_64) and the problem
is still there.
(BTW, in testing this I've been toggling the mds caps between 'allow
rw' and 'allow r, allow rw path=/k8s'. During this I've noticed that
the new caps are only applied after I remount.)
Cheers, Dan
On Fri, Nov 11, 2016 at 1:29 AM, Goncalo Borges
<goncalo.borges@xxxxxxxxxxxxx> wrote:
Hi Dan,,,
I know there are path restriction issues in the kernel client. See the discussion here.
http://lists.opennebula.org/pipermail/ceph-users-ceph.com/2016-June/010656.html
http://tracker.ceph.com/issues/16358
Cheers
Goncalo
________________________________________
From: ceph-users [ceph-users-bounces@xxxxxxxxxxxxxx] on behalf of Dan van der Ster [dan@xxxxxxxxxxxxxx]
Sent: 11 November 2016 02:41
To: ceph-users; Yan, Zheng
Subject: Intermittent permission denied using kernel client with mds path cap
Hi all, Hi Zheng,
We're seeing a strange issue with the kernel cephfs clients, combined
with a path restricted mds cap. It seems that files/dirs are
intermittently not created due to permission denied.
For example, when I untar a kernel into cephfs, we see ~1/1000 files
failed to open/mkdir.
Client caps are in the PS [1].
We've tried kernels 3.10.0-493.el7, 4.8.6, and 4.9-rc4 -- all have the
same intermittent behaviour. We could *not* reproduce the issue with
ceph-fuse 10.2.3.
The cluster is running 10.2.3.
Now, if we remove the path restricted cap -- i.e. use mds 'allow rw'
-- then we have no more errors.
So it seems there is a race in the path restriction cap code.
We grabbed an mds log, and noticed that it seems that a file is opened
twice, then the second open fails with 'already xlocked'. A full log
for one such file is here: http://pastebin.com/raw/YyULfjND
Is anyone successfully using path caps with kernel clients? Maybe this
is a new bug?
Cheers, Dan
[1]
[client.k8s]
key: xxx==
caps: [mds] allow r, allow rw path=/k8s
caps: [mon] allow r
caps: [osd] allow rw pool=cephfs_data_k8s
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
