Regards,
Zhi Zhang (David)
Contact: zhang.david2011@xxxxxxxxx
zhangz.david@xxxxxxxxxxx
On Thu, Mar 3, 2016 at 8:16 PM, Yan, Zheng <ukernel@xxxxxxxxx> wrote:
> On Mon, Feb 29, 2016 at 4:13 PM, Zhi Zhang <zhang.david2011@xxxxxxxxx> wrote:
>> Hi cepher,
>>
>> Steps to reproduce this issue:
>> 1. reduce mds_cache_size to 200 for quick verification
>> 2. mount sub directory as root_directory, e.g, ceph-fuse -r /zhi_test
>> /mnt/cephfs
>> 3. create a new sub directory under /mnt/cephfs, e.g, /mnt/cephfs/sub_dir
>> 4. write more than 200 files into /mnt/cephfs/sub_dir/ directory
>> 5. client should start to trim_caps, but caps are not released.
>>
>> ceph version: 0.94.1 with some backports
>> fuse version: both 2.8.3 or 2.9.2
>>
>> By further looking at the codes, ceph-fuse wants fuse to invalidate
>> some dentries by calling Client::_async_dentry_invalidate, which will
>> call dentry_invalidate_cb in fuse_ll.cc and fuse's API
>> fuse_lowlevel_notify_delete.
>>
>> Within these calls, the parent's inode (dirino) is our mount point's
>> inode (zhi_test), NOT the original default root inode ("/"), the child
>> inode may be a fake inode or sub directory's inode (sub_dir).
>>
>> However, fuse's API fuse_lowlevel_notify_delete returns -2, so no
>> ll_forget will be called on those files under sub_dir and those caps
>> are not released. MDS will report failing to respond to cache
>> pressure.
>>
>> We have changed the parent's inode to the original default root inode
>> ("/") in Client::_async_dentry_invalidate, but no changes on child
>> inode. After this, fuse can invalidate child dentry (sub_dir)
>> successfully and ll_forget will be called on those files under
>> sub_dir.
>>
>> I didn't find much information from the fuse's doc
>> https://fossies.org/dox/fuse-2.9.5/fuse__lowlevel_8h.html#a94893b3eb83fe2a218e80e43089670a1.
>> But it looks like fuse looking for the child from the head of the
>> tree, can't from an intermediate node? otherwise is it a bug from ceph
>> side?
>
> Using fuse_lowlevel_notify_delete() to invalidate dentry no longer
> works for 3.18+ kernel. See http://tracker.ceph.com/issues/9997 for
> detail
>
I am using centos-7 whose kernel is 3.10.x. I also tried to set
client_try_dentry_invalidate as false to force use remount to
invalidate dentry. It works but performance degrades.
fuse_lowlevel_notify_delete() works on kernel 3.10.x. What I found is
that the second args (fuse_ino_t parent) of this function should be
the ceph-fuse's root inode number ("/"), can't be others.
> Regards
> Yan, Zheng
>
>
>> Thanks.
>>
>> Regards,
>> Zhi Zhang (David)
>> Contact: zhang.david2011@xxxxxxxxx
>> zhangz.david@xxxxxxxxxxx
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
