On Wed, Jan 20, 2016 at 12:46 AM, Sage Weil <sweil@xxxxxxxxxx> wrote: > On Tue, 19 Jan 2016, Haomai Wang wrote: >> On Tue, Jan 19, 2016 at 11:15 PM, Sage Weil <sweil@xxxxxxxxxx> wrote: >> I think we should add an optional 'config' file location to the >> config >> search path. This would allow you to put customized options for >> an >> individual daemon in the daemon data directory. Or, you could >> avoid >> putting a config in /etc/ceph entirely, so that the daemon data >> is >> completely self-contained on the device. That would streamline >> boot disk >> failure (just reinstall OS and ceph package) or drive swaps >> between failed >> host hardware. >> >> What do you think? >> >> >> Totally agreed! >> >> Actually we have implemented alike thing downstream, ceph-osd,ceph-mon will >> check ceph.conf in the local dir(/var/lib/ceph/*/*/ceph.conf) then overwrite >> existing config values. That's said, it will load system ceph >> config(/etc/ceph/ceph.conf) file then load local ceph.conf. > > The one reservation about this direction is that there are a few devices > where I'm not sure how to attach the metadata (keyring, conf, and other > random bits in the osd_data dir) to the device: the SPDK NVMe driver, and > kinetic. These are somewhat obscure, but I'm curious if there are other > ideas floating around about how this should be approached... Just like BlueStore::_write_bdev_label, I think it should use BlockDevice::raw_write api which is used to record system metadata instead of bufferlist::write_fd. So keyring, conf and others could be reserved at the first section? But it may exists dependence loop, if we want to read config/keyring/device_info from osd, we need to know non-kernel device info at first then we can attach this and do reading. A possible solution is that we could recover info from non-kernel device. For example, we still store osd info at os directory(/var/lib/ceph/osd/ceph-0), but we will write these into device(including kernel or non-kernel), if os disk failed, we could use ceph-device-scan tool to discover this and extract necessary info from this disk into the active os directory? > > sage -- Best Regards, Wheat -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
