On 24/10/2013 03:09, Yan, Zheng wrote:
On Thu, Oct 24, 2013 at 6:44 AM, Michael <michael@xxxxxxxxxxxxxxxxxx> wrote:
Tying to gather some more info.
CentOS - hanging ls
[root@srv ~]# cat /proc/14614/stack
[<ffffffffa02d3e81>] wait_answer_interruptible+0x81/0xc0 [fuse]
[<ffffffffa02d415b>] fuse_request_send+0x1cb/0x290 [fuse]
[<ffffffffa02d652c>] fuse_do_getattr+0x10c/0x2c0 [fuse]
[<ffffffffa02d6755>] fuse_update_attributes+0x75/0x80 [fuse]
[<ffffffffa02d67b3>] fuse_getattr+0x53/0x60 [fuse]
[<ffffffff81186d51>] vfs_getattr+0x51/0x80
[<ffffffff81186de0>] vfs_fstatat+0x60/0x80
[<ffffffff81186f2b>] vfs_stat+0x1b/0x20
[<ffffffff81186f54>] sys_newstat+0x24/0x50
[<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
Ubuntu - hanging ls
root@srv:~# cat /proc/30012/stack
[<ffffffffa061d04b>] ceph_mdsc_do_request+0xcb/0x1a0 [ceph]
[<ffffffffa0608f37>] ceph_do_getattr+0xe7/0x120 [ceph]
[<ffffffffa0608f94>] ceph_getattr+0x24/0x100 [ceph]
[<ffffffff8118d42e>] vfs_getattr+0x4e/0x80
[<ffffffff8118d4ae>] vfs_fstatat+0x4e/0x70
[<ffffffff8118d4ee>] vfs_lstat+0x1e/0x20
[<ffffffff8118d68a>] sys_newlstat+0x1a/0x40
[<ffffffff816a6ba9>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
Started occurring shortly (within an hour or so) after adding a pool, not
sure if that's relevant yet.
-Michael
On 23/10/2013 21:10, Michael wrote:
I have a filesystem shared by several systems mounted on 2 ceph nodes with
a 3rd as a reference monitor.
It's been used for a couple of months now but suddenly the root directory
for the mount has become inaccessible and requests to files in it just hang,
there's no ceph errors reported before/after and subdirectories of the
directory can be used (and still are currently being used by VM's still
running from it). It's being mounted in a mixed kernel driver (ubuntu) and
centos (ceph-fuse) environment.
kernel, ceph-fuse and ceph-mds version? the hang was likely caused by an known
bug in kernel 3.10.
Regards
Yan, Zheng
Centos 6.4
2.6.32-358.23.2.el6.x86_64
ceph.x86_64 0.67.4-0.el6
ceph-fuse.x86_64 0.67.4-0.el6
Ubuntu 12.04
3.5.0-41-generic
Ceph Version: 0.67.2-1precise
... So it looks like I've let my ceph versions get out of sync, the MDS
is on a ubuntu box and all of the OSD are on Ubuntu boxes too, the
CentOS just has another MON on it, think I really drag myself away from
CentOS outright for ceph. I was previously using fuse on the Ubuntu
boxes as well, though that changed a few days ago (Currently walking
around the ceph features, next up was to hook ceph's rdb up to
OpenNebula, hence the additional pool).
-Michael
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
