Re: Client can't unmount, some pinned dentries in cache.

[Andrey Stepachev <octo47@xxxxxxxxx>]

Hi all.

I've drill down this bug. Ceph issues ->get() on inode in
ctor

dir_result_t::dir_result_t(Inode *in)
  : inode(in), offset(0), next_offset(2),
    release_count(0), start_shared_gen(0),
    buffer(0) {
  inode->get();
}

But don't deref in destructor. So it hard to
see, that problem can be in code like this

int Client::_opendir(Inode *in, dir_result_t **dirpp, int uid, int gid)
{
  *dirpp = new dir_result_t(in);
  if (!in->is_dir())
    return -ENOTDIR;
...
}

Patch attached.

2012/1/11 Andrey Stepachev <octo47@xxxxxxxxx>:
> Hi all.
>
> I have client hangs on unmount. Logs show, that
> some pinned entries in cache exists, but I can't understand
> what is going on and why they still referenced.
>
> 2012-01-11 15:29:27.168712 7f32f6ffd700 client.4915 unmounting: trim
> pass, size was 0+6
> 2012-01-11 15:29:27.168729 7f32f6ffd700 client.4915 unmounting: trim
> pass, size still 0+6
> 2012-01-11 15:29:27.168751 7f32f6ffd700 client.4915 dump_cache: inode
> 10000007dc3.head ref 1 dir 0 10000007dc3.head(ref=1
> cap_refs={1024=0,2048=0} open={1=0} mode=100777 size=734
> mtime=2012-01-11 15:13:53.624282 caps=p(0=p) objectset[10000007dc3 ts
> 0/0 objects 1 dirty_or_tx 0] 0x1210d80)
> 2012-01-11 15:29:27.168767 7f32f6ffd700 client.4915 dump_cache: inode
> 10000007586.head ref 1 dir 0 10000007586.head(ref=1
> cap_refs={1024=0,2048=0} open={1=0} mode=100777 size=714
> mtime=2012-01-11 12:39:00.960535 caps=p(0=p) objectset[10000007586 ts
> 0/0 objects 1 dirty_or_tx 0] 0x166d370)
> 2012-01-11 15:29:27.168779 7f32f6ffd700 client.4915 dump_cache: inode
> 1000000758d.head ref 1 dir 0 1000000758d.head(ref=1
> cap_refs={1024=0,2048=0} open={1=0} mode=100777 size=734
> mtime=2012-01-11 15:25:09.622986 caps=p(0=p) objectset[1000000758d ts
> 0/0 objects 1 dirty_or_tx 0] 0x12111c0)
> 2012-01-11 15:29:27.168791 7f32f6ffd700 client.4915 dump_cache: inode
> 100000079df.head ref 2 dir 0 100000079df.head(ref=2
> cap_refs={1024=0,2048=0,4096=0,8192=0} open={1=0,2=0} mode=100777
> size=1089 mtime=2012-01-11 15:26:23.780076 caps=p(0=p)
> objectset[100000079df ts 0/0 objects 1 dirty_or_tx 0] 0x12a99b0)
> 2012-01-11 15:29:27.168803 7f32f6ffd700 client.4915 dump_cache: inode
> 100000079e0.head ref 2 dir 0 100000079e0.head(ref=2
> cap_refs={1024=0,2048=0,4096=0,8192=0} open={1=0,2=0} mode=100777
> size=734 mtime=2012-01-11 15:27:13.887592 caps=p(0=p)
> objectset[100000079e0 ts 0/0 objects 0 dirty_or_tx 0] 0x7f32f833cb00)
> 2012-01-11 15:29:27.168814 7f32f6ffd700 client.4915 dump_cache: inode
> 100000079d7.head ref 1 dir 0 100000079d7.head(ref=1
> cap_refs={1024=0,2048=0} open={1=0} mode=100777 size=734
> mtime=2012-01-11 12:40:14.325753 caps=p(0=p) objectset[100000079d7 ts
> 0/0 objects 1 dirty_or_tx 0] 0x1211750)
>
> --
> Andrey.



-- 
Андрей.

From 76b8c85ddf059a971e903fbe25d469d1798be115 Mon Sep 17 00:00:00 2001
From: Andrey Stepachev <octo@xxxxxxxxxxxxxx>
Date: Thu, 12 Jan 2012 19:26:34 +0400
Subject: [PATCH] avoid ref counter increment in case of nonexistent dir

---
 src/client/Client.cc |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/src/client/Client.cc b/src/client/Client.cc
index 0aed0b2..c0cebcb 100644
--- a/src/client/Client.cc
+++ b/src/client/Client.cc
@@ -4140,9 +4140,9 @@ int Client::opendir(const char *relpath, dir_result_t **dirpp)
 
 int Client::_opendir(Inode *in, dir_result_t **dirpp, int uid, int gid) 
 {
-  *dirpp = new dir_result_t(in);
   if (!in->is_dir())
     return -ENOTDIR;
+  *dirpp = new dir_result_t(in);
   (*dirpp)->set_frag(in->dirfragtree[0]);
   if (in->dir)
     (*dirpp)->release_count = in->dir->release_count;
-- 
1.7.5.4