On 2015-07-03 01:31:35 +0000, Johannes Formann said:
Hi,
When rebooting one of the nodes (e. g. for a kernel upgrade) the OSDs
do not seem to shut down correctly. Clients hang and ceph osd tree show
the OSDs of that node still up. Repeated runs of ceph osd tree show
them going down after a while. For instance, here OSD.7 is still up,
even though the machine is in the middle of the reboot cycle.
...
Any ideas as to what is causing this or how to diagnose this?
I see this behavior (only) when I reboot a ceph-node with a monitor and OSDs.
I guess somehow this relates. (OSD-messages getting lost due to the
„failing“ mon)
Sorry for being silent for a few days, other things kept me busy.
Indeed this an interesting thought. We do have MONs running on three of
our storage nodes. I need to verify if the one where I aw the problem
is one of them, but with 5 total, there is more than 50% chance ;)
Can any tell me which log levels on the MONs and/or OSDs I might want
to change to track if the shutdown notification are actually received
by the monitors or where they get lost?
Regards,
Daniel
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
