Thank you Eric and Ilya, The disk crash does not seems to not be related even though it happened almost at the same time then I guess. No dd until 3.15.9 then. ;) Thank you again. On Tue, Aug 5, 2014 at 4:28 PM, Ilya Dryomov <ilya.dryomov@xxxxxxxxxxx> wrote: > On Tue, Aug 5, 2014 at 5:59 PM, Thorwald Lundqvist > <thorwald@xxxxxxxxxxxxxx> wrote: >> Hi! >> >> I've just experienced a disk crash in my ceph cluster. This seems to >> have caused an error in the kernel since every I/O command sent to any >> mapped RBD is defered indefinitely, even an hour later, the OSD is >> already taken out of the cluster and ceph -s says HEALTH_OK. >> >> A kernel dump is provided at this pastebin link: http://pastebin.com/WMqmiUsM >> >> What did I do prior the crash? >> >> I mapped one clean rbd device at /dev/rbd0 and one rbd device that had >> a partition and ext4 filesystem at 20GB each >> >> I started to dd rbd1 to rbd0 (dd if=/dev/rbd1 of=/dev/rbd0 bs=4M) It >> copied a few GB, then deep-scrub started and at that moment one of the >> disks failed. and dd got stuck (disk defered). And it still is. >> >> I think that only a reboot of the server can fix this problem now. >> >> This is not normal behavior when a disk crashes, right? > > No, it's not ;) Technically this is not a crash though, it's more of > a deadlock - one process ended up blocking forever while holding > a crucial lock. See http://tracker.ceph.com/issues/8818. > > The fix is going to be in 3.17-rc1 so it should catch the last 3.15 > stable update. > > Thanks, > > Ilya -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
