On Thu, Jul 6, 2017 at 2:23 PM, Stanislav Kopp <staskopp@xxxxxxxxx> wrote: > 2017-07-06 14:16 GMT+02:00 Ilya Dryomov <idryomov@xxxxxxxxx>: >> On Thu, Jul 6, 2017 at 1:28 PM, Stanislav Kopp <staskopp@xxxxxxxxx> wrote: >>> Hi, >>> >>> 2017-07-05 20:31 GMT+02:00 Ilya Dryomov <idryomov@xxxxxxxxx>: >>>> On Wed, Jul 5, 2017 at 7:55 PM, Stanislav Kopp <staskopp@xxxxxxxxx> wrote: >>>>> Hello, >>>>> >>>>> I have problem that sometimes I can't unmap rbd device, I get "sysfs >>>>> write failed rbd: unmap failed: (16) Device or resource busy", there >>>>> is no open files and "holders" directory is empty. I saw on the >>>>> mailling list that you can "force" unmapping the device, but I cant >>>>> find how does it work. "man rbd" only mentions "force" as "KERNEL RBD >>>>> (KRBD) OPTION", but "modinfo rbd" doesn't show this option. Did I miss >>>>> something? >>>> >>>> Forcing unmap on an open device is not a good idea. I'd suggest >>>> looking into what's holding the device and fixing that instead. >>> >>> We use pacemaker's resource agent for rbd mount/unmount >>> (https://github.com/ceph/ceph/blob/master/src/ocf/rbd.in) >>> I've reproduced the failure again and now saw in ps output that there >>> is still unmout fs process in D state: >>> >>> root 29320 0.0 0.0 21980 1272 ? D 09:18 0:00 >>> umount /export/rbd1 >>> >>> this explains rbd unmap problem, but strange enough I don't see this >>> mount in /proc/mounts, so it looks like it was successfully unmounted, >>> if I try to strace the "umount" procces it hung (the strace, with no >>> output), looks like kernel problem? Do you have some tips for further >>> debugging? >> >> Check /sys/kernel/debug/ceph/<cluster-fsid.client-id>/osdc. It lists >> in-flight requests, that's what umount is blocked on. > > I see this in my output, but don't know what does it means honestly: > > root@nfs-test01:~# cat > /sys/kernel/debug/ceph/4f23f683-21e6-49f3-ae2c-c95b150b9dc6.client138566/osdc > REQUESTS 2 homeless 0 > 658 osd9 0.75514984 [9,1,6]/9 [9,1,6]/9 > rbd_data.6e28c6b8b4567.0000000000000000 0x400024 10'0 > set-alloc-hint,write > 659 osd15 0.40f1ea02 [15,7,9]/15 [15,7,9]/15 > rbd_data.6e28c6b8b4567.0000000000000001 0x400024 10'0 > set-alloc-hint,write It means you have two pending writes (OSD requests), to osd9 and osd15. What is the output of $ ceph -s $ ceph pg dump pgs_brief Thanks, Ilya _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |