Hey folks, Ran into this today. Not sure what I did wrong. =) I had an RBD successfully mounted and was done with it. Proceeded to do the following: root@spcnode2:~# ls /sys/bus/rbd/devices/ 0 root@spcnode2:~# echo 0 > /sys/bus/rbd/remove root@spcnode2:~# ls /sys/bus/rbd/devices/ <--- At this point, I believe the RBD has been successfully removed ---- About an hour passes where I am messing with my ceph cluster. No other commands are run on this machine ---- ---- New cluster is up. Time to mount my new RBD root@spcnode2:~# echo "10.55.30.0,10.55.30.1,10.55.30.2 name=admin,secret=AQCNv+BPoPQENBAAxlm39kJ5XteNxg2S/dulXw== rbd perftest" | tee /sys/bus/rbd/add 10.55.30.0,10.55.30.1,10.55.30.2 name=admin,secret=AQCNv+BPoPQENBAAxlm39kJ5XteNxg2S/dulXw== rbd perftest Segmentation fault Well that's ugly. What's in syslog? Jun 19 11:16:56 spcnode2 kernel: [76564.387890] ------------[ cut here ]------------ Jun 19 11:16:56 spcnode2 kernel: [76564.392569] WARNING: at /build/buildd/linux-3.2.0/fs/sysfs/inode.c:324 sysfs_hash_and_remove+0xa9/0xb0() Jun 19 11:16:56 spcnode2 kernel: [76564.402233] Hardware name: Relion 1702 Jun 19 11:16:56 spcnode2 kernel: [76564.406079] sysfs: can not remove 'bdi', no directory Jun 19 11:16:56 spcnode2 kernel: [76564.411268] Modules linked in: rbd libceph ip6table_filter ip6_tables ebtable_nat ebtables ipt_MASQUERADE xt_state ipt_REJECT xt_CHECKSUM iptable_mangle xt_tcpudp xt_conntrack iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 ipmi_devintf ipmi_si iptable_filter ipmi_msghandler ip_tables x_tables kvm_intel kvm bnep rfcomm bluetooth parport_pc ppdev nfsd nfs lockd fscache auth_rpcgss nfs_acl sunrpc ext2 xfs vesafb ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi bridge mtdchar i7core_edac psmouse 8021q garp stp lp parport dm_multipath mac_hid serio_raw edac_core ioatdma usbhid hid sfc mtd i2c_algo_bit igb mdio dca btrfs zlib_deflate libcrc32c Jun 19 11:16:56 spcnode2 kernel: [76564.477972] Pid: 6924, comm: bash Tainted: G D W 3.2.0-25-generic #40-Ubuntu Jun 19 11:16:56 spcnode2 kernel: [76564.485837] Call Trace: Jun 19 11:16:56 spcnode2 kernel: [76564.488394] [<ffffffff810672af>] warn_slowpath_common+0x7f/0xc0 Jun 19 11:16:56 spcnode2 kernel: [76564.494511] [<ffffffff810673a6>] warn_slowpath_fmt+0x46/0x50 Jun 19 11:16:56 spcnode2 kernel: [76564.500348] [<ffffffff81192958>] ? iput_final+0xe8/0x210 Jun 19 11:16:56 spcnode2 kernel: [76564.505888] [<ffffffff811ebc59>] sysfs_hash_and_remove+0xa9/0xb0 Jun 19 11:16:56 spcnode2 kernel: [76564.512082] [<ffffffff811ee356>] sysfs_remove_link+0x26/0x30 Jun 19 11:16:56 spcnode2 kernel: [76564.517959] [<ffffffff812fb960>] del_gendisk+0x100/0x260 Jun 19 11:16:56 spcnode2 kernel: [76564.523448] [<ffffffffa0623868>] rbd_dev_release+0x108/0x110 [rbd] Jun 19 11:16:56 spcnode2 kernel: [76564.529861] [<ffffffff813f1407>] device_release+0x27/0xa0 Jun 19 11:16:56 spcnode2 kernel: [76564.535432] [<ffffffff8130cfdc>] kobject_release+0x4c/0xa0 Jun 19 11:16:56 spcnode2 kernel: [76564.541163] [<ffffffff8130cf90>] ? kobject_del+0x40/0x40 Jun 19 11:16:56 spcnode2 kernel: [76564.546694] [<ffffffff8130e686>] kref_put+0x36/0x70 Jun 19 11:16:56 spcnode2 kernel: [76564.551764] [<ffffffff8130ce97>] kobject_put+0x27/0x60 Jun 19 11:16:56 spcnode2 kernel: [76564.557126] [<ffffffff8131d33c>] ? _kstrtoull+0x2c/0x90 Jun 19 11:16:56 spcnode2 kernel: [76564.562523] [<ffffffff813f1167>] put_device+0x17/0x20 Jun 19 11:16:56 spcnode2 kernel: [76564.567808] [<ffffffff813f225e>] device_unregister+0x1e/0x30 Jun 19 11:16:56 spcnode2 kernel: [76564.573647] [<ffffffffa06211ea>] rbd_remove+0x15a/0x160 [rbd] Jun 19 11:16:56 spcnode2 kernel: [76564.579594] [<ffffffff813f3c47>] bus_attr_store+0x27/0x30 Jun 19 11:16:56 spcnode2 kernel: [76564.585113] [<ffffffff811ebebf>] sysfs_write_file+0xef/0x170 Jun 19 11:16:56 spcnode2 kernel: [76564.590907] [<ffffffff81177f23>] vfs_write+0xb3/0x180 Jun 19 11:16:56 spcnode2 kernel: [76564.596158] [<ffffffff8117824a>] sys_write+0x4a/0x90 Jun 19 11:16:56 spcnode2 kernel: [76564.601258] [<ffffffff81665c42>] system_call_fastpath+0x16/0x1b Jun 19 11:16:56 spcnode2 kernel: [76564.607321] ---[ end trace ace27f1cbf93eeaa ]--- Jun 19 11:16:57 spcnode2 kernel: [76564.612447] BUG: unable to handle kernel NULL pointer dereference at 0000000000000079 Jun 19 11:16:57 spcnode2 kernel: [76564.620374] IP: [<ffffffff811ed770>] sysfs_find_dirent+0x10/0x110 Jun 19 11:16:57 spcnode2 kernel: [76564.626475] PGD 404514067 PUD 5f89cc067 PMD 0 Jun 19 11:16:57 spcnode2 kernel: [76564.630958] Oops: 0000 [#2] SMP Jun 19 11:16:57 spcnode2 kernel: [76564.634254] CPU 5 Jun 19 11:16:57 spcnode2 kernel: [76564.636113] Modules linked in: rbd libceph ip6table_filter ip6_tables ebtable_nat ebtables ipt_MASQUERADE xt_state ipt_REJECT xt_CHECKSUM iptable_mangle xt_tcpudp xt_conntrack iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 ipmi_devintf ipmi_si iptable_filter ipmi_msghandler ip_tables x_tables kvm_intel kvm bnep rfcomm bluetooth parport_pc ppdev nfsd nfs lockd fscache auth_rpcgss nfs_acl sunrpc ext2 xfs vesafb ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi bridge mtdchar i7core_edac psmouse 8021q garp stp lp parport dm_multipath mac_hid serio_raw edac_core ioatdma usbhid hid sfc mtd i2c_algo_bit igb mdio dca btrfs zlib_deflate libcrc32c Jun 19 11:16:57 spcnode2 kernel: [76564.701251] Jun 19 11:16:57 spcnode2 kernel: [76564.702740] Pid: 6924, comm: bash Tainted: G D W 3.2.0-25-generic #40-Ubuntu Penguin Computing Relion 1702/X8DTT Jun 19 11:16:57 spcnode2 kernel: [76564.713752] RIP: 0010:[<ffffffff811ed770>] [<ffffffff811ed770>] sysfs_find_dirent+0x10/0x110 Jun 19 11:16:57 spcnode2 kernel: [76564.722319] RSP: 0018:ffff8805f8f9bc58 EFLAGS: 00010246 Jun 19 11:16:57 spcnode2 kernel: [76564.727719] RAX: ffff8806186edbc0 RBX: 0000000000000000 RCX: 00000000000988e6 Jun 19 11:16:57 spcnode2 kernel: [76564.734892] RDX: ffffffff81a0158d RSI: 0000000000000000 RDI: 0000000000000000 Jun 19 11:16:57 spcnode2 kernel: [76564.742083] RBP: ffff8805f8f9bc78 R08: ffffea00303f6580 R09: ffffffff8130cfe9 Jun 19 11:16:57 spcnode2 kernel: [76564.749221] R10: ffff880c0fe5de28 R11: 0000000000000000 R12: 0000000000000000 Jun 19 11:16:57 spcnode2 kernel: [76564.756437] R13: ffffffff81a0158d R14: ffff880bf45a5a50 R15: ffff880c0fd1de18 Jun 19 11:16:57 spcnode2 kernel: [76564.763630] FS: 00007fe308eb7700(0000) GS:ffff880c3fc20000(0000) knlGS:0000000000000000 Jun 19 11:16:57 spcnode2 kernel: [76564.771717] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jun 19 11:16:57 spcnode2 kernel: [76564.777549] CR2: 0000000000000079 CR3: 00000005f89cd000 CR4: 00000000000006e0 Jun 19 11:16:57 spcnode2 kernel: [76564.784738] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Jun 19 11:16:57 spcnode2 kernel: [76564.791877] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Jun 19 11:16:57 spcnode2 kernel: [76564.798991] Process bash (pid: 6924, threadinfo ffff8805f8f9a000, task ffff8806186edbc0) Jun 19 11:16:57 spcnode2 kernel: [76564.807295] Stack: Jun 19 11:16:57 spcnode2 kernel: [76564.809302] 0000000000000000 0000000000000000 ffffffff81a0158d ffff880bf45a5a50 Jun 19 11:16:57 spcnode2 kernel: [76564.816832] ffff8805f8f9bca8 ffffffff811ed9bc ffff8805f8f9bcd8 ffffffff81c34b00 Jun 19 11:16:57 spcnode2 kernel: [76564.824341] ffff880605b36878 0000000000000000 ffff8805f8f9bce8 ffffffff811efa15 Jun 19 11:16:57 spcnode2 kernel: [76564.831894] Call Trace: Jun 19 11:16:57 spcnode2 kernel: [76564.834337] [<ffffffff811ed9bc>] sysfs_get_dirent+0x3c/0x80 Jun 19 11:16:57 spcnode2 kernel: [76564.840041] [<ffffffff811efa15>] sysfs_remove_group+0x35/0x100 Jun 19 11:16:57 spcnode2 kernel: [76564.846029] [<ffffffff810fee24>] blk_trace_remove_sysfs+0x14/0x20 Jun 19 11:16:57 spcnode2 kernel: [76564.852195] [<ffffffff812f50d9>] blk_unregister_queue+0x59/0x80 Jun 19 11:16:57 spcnode2 kernel: [76564.858270] [<ffffffff812fb97b>] del_gendisk+0x11b/0x260 Jun 19 11:16:57 spcnode2 kernel: [76564.863661] [<ffffffffa0623868>] rbd_dev_release+0x108/0x110 [rbd] Jun 19 11:16:57 spcnode2 kernel: [76564.869962] [<ffffffff813f1407>] device_release+0x27/0xa0 Jun 19 11:16:57 spcnode2 kernel: [76564.875448] [<ffffffff8130cfdc>] kobject_release+0x4c/0xa0 Jun 19 11:16:57 spcnode2 kernel: [76564.881061] [<ffffffff8130cf90>] ? kobject_del+0x40/0x40 Jun 19 11:16:57 spcnode2 kernel: [76564.886502] [<ffffffff8130e686>] kref_put+0x36/0x70 Jun 19 11:16:57 spcnode2 kernel: [76564.891521] [<ffffffff8130ce97>] kobject_put+0x27/0x60 Jun 19 11:16:57 spcnode2 kernel: [76564.896739] [<ffffffff8131d33c>] ? _kstrtoull+0x2c/0x90 Jun 19 11:16:57 spcnode2 kernel: [76564.902043] [<ffffffff813f1167>] put_device+0x17/0x20 Jun 19 11:16:57 spcnode2 kernel: [76564.907226] [<ffffffff813f225e>] device_unregister+0x1e/0x30 Jun 19 11:16:57 spcnode2 kernel: [76564.913057] [<ffffffffa06211ea>] rbd_remove+0x15a/0x160 [rbd] Jun 19 11:16:57 spcnode2 kernel: [76564.918881] [<ffffffff813f3c47>] bus_attr_store+0x27/0x30 Jun 19 11:16:57 spcnode2 kernel: [76564.924436] [<ffffffff811ebebf>] sysfs_write_file+0xef/0x170 Jun 19 11:16:57 spcnode2 kernel: [76564.930174] [<ffffffff81177f23>] vfs_write+0xb3/0x180 Jun 19 11:16:57 spcnode2 kernel: [76564.935450] [<ffffffff8117824a>] sys_write+0x4a/0x90 Jun 19 11:16:57 spcnode2 kernel: [76564.940497] [<ffffffff81665c42>] system_call_fastpath+0x16/0x1b Jun 19 11:16:57 spcnode2 kernel: [76564.946488] Code: 41 5c 41 5d 41 5e 41 5f 5d c3 90 4c 89 f7 e8 68 df 46 00 eb c3 0f 0b 0f 1f 40 00 55 48 89 e5 41 56 41 55 41 54 53 66 66 66 66 90 <80> 7f 79 00 4c 8b 67 70 49 89 d6 48 89 f3 0f 95 c0 48 85 f6 0f Jun 19 11:16:57 spcnode2 kernel: [76564.966571] RIP [<ffffffff811ed770>] sysfs_find_dirent+0x10/0x110 Jun 19 11:16:57 spcnode2 kernel: [76564.972826] RSP <ffff8805f8f9bc58> Jun 19 11:16:57 spcnode2 kernel: [76564.976331] CR2: 0000000000000079 Jun 19 11:16:57 spcnode2 kernel: [76564.979725] ---[ end trace ace27f1cbf93eeab ]--- Had to do a hard reset on the machine afterwards. The machine mounting the RBD is running Ubuntu 12.04, and is not hosting any OSDs or MONs. root@spcnode2:~# uname -a Linux spcnode2 3.2.0-25-generic #40-Ubuntu SMP Wed May 23 20:30:51 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux root@spcnode2:~# ceph --version ceph version 0.47.2 (commit:8bf9fde89bd6ebc4b0645b2fe02dadb1c17ad372) - Travis -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
