On Fri, 25 Oct 2013, Loic Dachary wrote: > Hi, > > If a disk containing an OSD is moved to another host, it is entirely > possible that the objects it contains are violating a crush rule > imposing that no object is replicated on the same host. Will this be > taken care of as a side effect of updating the place of the OSD in the > map ( > https://github.com/ceph/ceph/blob/v0.61.9/src/upstart/ceph-osd.conf#L29 > ) ? Exactly. In general, in the simlest case, plugging an OSD into a different part of the CRUSH map will mean that the data will move. The benefit is that we retain that copy so there is no spike in the probability of losing additional replicas and all your data. There are some tricks we could play if a host is being replaced wholesale (all drives moved together) by resuing the internal CRUSH ids and preventing movement, but that isn't done yet. sage > > Cheers > > On 23/10/2013 15:45, Loic Dachary wrote: > > Hi, > > > > As a conclusion to this thread and also because I realized that most people are unaware of the level of automation provided by ceph / udev, I just posted a quick summary : http://dachary.org/?p=2428 > > > > Cheers > > > > On 22/10/2013 19:31, Sage Weil wrote: > >> On Tue, 22 Oct 2013, Gregory Farnum wrote: > >>> On Mon, Oct 21, 2013 at 11:13 PM, Loic Dachary <loic@xxxxxxxxxxx> wrote: > >>>> > >>>> > >>>> On 21/10/2013 18:49, Gregory Farnum wrote: > >>>>> I'm not quite sure what questions you're actually asking here... > >>>>> In general, the OSD is not removed from the system without explicit > >>>>> admin intervention. When it is removed, all traces of it should be > >>>>> zapped (including its key), so it can't reconnect. > >>>>> If it hasn't been removed, then indeed it will continue working > >>>>> properly even if moved to a different box. > >>>> > >>>> If there is an external journal, the device containing the journal needs to be moved with the device containing the data. If I read ceph/src/upstart/ceph-osd.conf correctly, when the data device is plugged in the new machine it will fail to start because the journal is not there yet. When the journal device is plugged in, the ceph-osd.conf would be called because udev rule in ceph/udev/95-ceph-osd.rules call ceph-disk activate-journal. > >>>> > >>>> Is my understanding correct ? > >>> > >>> Well, after being wrong last time I'm a little reluctant to make > >>> pronouncements from memory, but that definitely sounds correct to me. > >> > >> Yep, that's how it's supposed to work. The activate-journal piece is > >> somewhat recent though (I think maybe it wasn't in place for cuttlefish?). > >> > >>> :) If I were doing an audit I'd want to look at what happens if there > >>> is a wrong journal in the correct location, etc. > >> > >> The ceph-osd will fail on start because the uuid/fsid doesn't match. > >> > >> sage > >> > > > > -- > Lo?c Dachary, Artisan Logiciel Libre > All that is necessary for the triumph of evil is that good people do nothing. > > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
