Please use a v3.4 kernel or later; there are many bugs in the 3.2 kernel and the fixes have not been backported that far. Thanks! sage On Wed, 19 Jun 2013, peter@xxxxxxxxx wrote: > Hi, > > We ran into the following problem: > > [ 194.789476] libceph: loaded (mon/osd proto 15/24, osdmap 5/6 5/6) > [ 194.798526] ceph: loaded (mds proto 32) > [ 194.800431] libceph: client0 fsid 97e515bb-d334-4fa7-8b53-7d85615809fd > [ 194.802534] libceph: mon0 10.255.0.25:6789 session established > [ 338.004303] BUG: unable to handle kernel NULL pointer dereference at > (null) > [ 338.005461] IP: [<ffffffffa025d0c2>] ceph_d_prune+0x22/0x30 [ceph] > [ 338.006277] PGD 1378e8067 PUD 13726e067 PMD 0 > [ 338.006918] Oops: 0002 [#1] SMP > [ 338.007386] CPU 0 > [ 338.007648] Modules linked in: ceph libceph libcrc32c ip6table_filter > ip6_tables iptable_filter ip_tables ebtable_nat ebtables x_tables > openvswitch(O) kvm_intel kvm nbd ib_iser rdma_cm ib_cm iw_cm ib_sa ib > _mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi > vesafb joydev i7core_edac mac_hid psmouse dm_multipath edac_core shpchp lp > serio_raw parport usbhid hid e1000e > [ 338.013103] > [ 338.013300] Pid: 5461, comm: umount Tainted: G O 3.2.0-44-generic > #69-Ubuntu Supermicro X8SIU/X8SIU > [ 338.014634] RIP: 0010:[<ffffffffa025d0c2>] [<ffffffffa025d0c2>] > ceph_d_prune+0x22/0x30 [ceph] > [ 338.015318] RSP: 0018:ffff88013822be08 EFLAGS: 00010286 > [ 338.015616] RAX: 0000000000000000 RBX: ffff8801214d7cc0 RCX: > 0000000100002529 > [ 338.016524] RDX: ffffffff81c31c00 RSI: ffffffff8118f220 RDI: > ffff8801214d7cc0 > [ 338.017438] RBP: ffff88013822be08 R08: 0000000000000001 R09: > 0000000000000000 > [ 338.018350] R10: 0000000000000000 R11: 0000000000000001 R12: > ffff8801214d7d40 > [ 338.019247] R13: 00000000018347f0 R14: 0000000000000000 R15: > 0000000000000000 > [ 338.020200] FS: 00007f17b5770800(0000) GS:ffff88013fc00000(0000) > knlGS:0000000000000000 > [ 338.021240] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 338.021978] CR2: 0000000000000000 CR3: 0000000137881000 CR4: > 00000000000006f0 > [ 338.022893] DR0: 0000000000000000 DR1: 0000000000000000 DR2: > 0000000000000000 > [ 338.023809] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: > 0000000000000400 > [ 338.024723] Process umount (pid: 5461, threadinfo ffff88013822a000, task > ffff88013417ae00) > [ 338.025802] Stack: > [ 338.026066] ffff88013822be28 ffffffff8118e8e7 ffff8801214d7cc0 > ffff8801214d7cc0 > [ 338.027112] ffff88013822be58 ffffffff8118f89a ffff88013822be18 > ffff88013822be18 > [ 338.028162] ffff8801340fd400 ffffffffa02730e0 ffff88013822be78 > ffffffff811918a9 > [ 338.029209] Call Trace: > [ 338.029539] [<ffffffff8118e8e7>] dentry_lru_prune+0x87/0x90 > [ 338.030275] [<ffffffff8118f89a>] > shrink_dcache_for_umount_subtree+0x7a/0x1e0 > [ 338.031187] [<ffffffff811918a9>] shrink_dcache_for_umount+0x49/0x60 > [ 338.031785] [<ffffffff8117bc8c>] generic_shutdown_super+0x2c/0xe0 > [ 338.032145] [<ffffffff8117bdd6>] kill_anon_super+0x16/0x30 > [ 338.032807] [<ffffffffa02590c0>] ceph_kill_sb+0x30/0x50 [ceph] > [ 338.033573] [<ffffffff8117c41c>] deactivate_locked_super+0x3c/0xa0 > [ 338.034379] [<ffffffff8117cc9e>] deactivate_super+0x4e/0x70 > [ 338.035116] [<ffffffff8119945d>] mntput_no_expire+0x9d/0xf0 > [ 338.035832] [<ffffffff8119a78b>] sys_umount+0x5b/0xd0 > [ 338.036493] [<ffffffff81667cc2>] system_call_fastpath+0x16/0x1b > [ 338.037267] Code: 66 90 b8 01 00 00 00 5d c3 55 48 89 e5 66 66 66 66 90 48 > 8b 47 18 48 85 c0 74 14 48 39 c7 74 0f 48 83 7f 10 00 74 08 48 8b 40 78 <f0> > 80 20 fd 5d c3 0f 1f 84 00 00 00 00 00 55 48 89 e5 48 > 83 ec > [ 338.041328] RIP [<ffffffffa025d0c2>] ceph_d_prune+0x22/0x30 [ceph] > [ 338.042165] RSP <ffff88013822be08> > [ 338.042613] CR2: 0000000000000000 > [ 338.043140] ---[ end trace 17d0a061021b6835 ]--- > > I can see that there is an old bug report here: > > http://tracker.ceph.com/issues/3088 > and perhaps > http://tracker.ceph.com/issues/2444 > > We are running 0.61.3 on Ubuntu 12.04 > > Kernel: > > root@xxx:~# uname -a > Linux xxxx 3.2.0-44-generic #69-Ubuntu SMP Thu May 16 17:35:01 UTC 2013 x86_64 > x86_64 x86_64 GNU/Linux > > The machine needs a reboot in order to mount cephfs again. We can easily > reproduce this if needed. > > Thanks, > > Peter > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com