On Mon, Nov 13, 2017 at 6:20 PM, Kyle Bader <kyle.bader@xxxxxxxxx> wrote: > Configuration files are often driven by configuration management, with > previous versions stored in some kind of version control systems. We > should make sure that if configuration moves to the monitors that you > have some form of history and rollback capabilities. It might be worth > modeling it similar to network switch configuration shells, a la > Junos. > > * change configuration > * require commit configuration change > * ability to rollback N configuration changes > * ability to diff to configuration versions > > That way an admin can figure out when the last configuration change > was, what changed, and rollback if necessary. That is an extremely good idea. As a minimal thing, it should be pretty straightforward to implement a snapshot/rollback. I imagine many users today are not so disciplined as to version control their configs, but this is a good opportunity to push that as the norm by building it in. John > > > On Mon, Nov 13, 2017 at 8:45 AM, Mark Nelson <mnelson@xxxxxxxxxx> wrote: >> >> >> On 11/13/2017 04:00 AM, John Spray wrote: >>> >>> On Mon, Nov 13, 2017 at 4:30 AM, Christian Wuerdig >>> <christian.wuerdig@xxxxxxxxx> wrote: >>>> >>>> Hm, have you guys considered utilizing existing key-value stores like >>>> Consul or etcd for this instead of rolling your own? Not sure about >>>> the details of etcd but the Consul API is quite nice, supports long >>>> polling and transactional support. Obvious downside is that you depend >>>> on a separate service but that can also be an advantage. >>> >>> >>> When it comes to putting and getting values, Consul and etcd don't >>> really offer much that the ceph mons don't already do. As you say, it >>> would be a new dependency, but more importantly it would also be a >>> whole new network comms path with its own authentication, ports, etc. >>> >>> This is one of those situations where using something off the shelf is >>> actually way more effort (for developers and for users) than building >>> it in. >>> >>> John >>> >> >> I don't disagree, but I could imagine there are a number of sysadmins that >> want Ceph to play nice with whatever they are currently using for everything >> else they maintain. Whatever we do here, we probably want to be mindful (ie >> I'd argue that deprecating ceph.conf might not be well liked by folks that >> are happy with their current setup). >> >> Mark >> >> >>>> >>>> On Sat, Nov 11, 2017 at 4:30 AM, Sage Weil <sweil@xxxxxxxxxx> wrote: >>>>> >>>>> I've started on this long-discussed feature! I haven't gotten too far >>>>> but >>>>> you can see what's there so far at >>>>> >>>>> https://github.com/ceph/ceph/pull/18856 >>>>> >>>>> The first thing perhaps is to finalize what flexibility we want to >>>>> support. I've a quick summary at >>>>> >>>>> http://pad.ceph.com/p/config >>>>> >>>>> Namely, >>>>> >>>>> config/option = value # like [global] >>>>> config/$type/option = value # like [mon] >>>>> config/$type.$id/option = value # like [mon.a] >>>>> >>>>> There are two new things: >>>>> >>>>> config/.../class:$classname/option = value >>>>> >>>>> For OSDs, this matches the device_class. So you can do something like >>>>> >>>>> config/osd/class:ssd/bluestore_cache_size = 10485760 # 10gb, woohoo! >>>>> >>>>> You can also match the crush location: >>>>> >>>>> config/.../$crushtype:$crushvalue/option = value >>>>> >>>>> e.g., >>>>> >>>>> config/osd/rack:foo/debug_osd = 10 # hunting some issue >>>>> >>>>> This obviously makes sense for OSDs. We can also make it makes sense >>>>> for >>>>> non-OSDs since everybody (clients and daemons) has a concept of >>>>> crush_location that is a set of key/value pairs like "host=foo rack=bar" >>>>> which match the CRUSH hierarchy. In this case, my plan is to make the >>>>> initial mon authentication step include the hostname of the host you're >>>>> connecting from and then extract the rest of the location by lookup >>>>> up the host in the CRUSH map. >>>>> >>>>> The precedence for these is described here: >>>>> >>>>> >>>>> https://github.com/ceph/ceph/pull/18856/commits/5abbd0c9e279022f185787238d21eabbbe28e336#diff-344645b5339d494e1839ff1fcaa5cb7dR15 >>>>> >>>>> >>>>> Lots of other thorny issues to consider. For example: >>>>> >>>>> - What about monitor configs? If they store their config paxos, and you >>>>> set an option that breaks paxos, how can you change/fix it? For the >>>>> moment I'm just ignoring the mons. >>>>> >>>>> - What about ceph.conf? My thought here is to mark which options are >>>>> legal for bootstrap (i.e., used during the initial connection to mon to >>>>> authenticate and fetch config), and warn on anything other than that in >>>>> ceph.conf. But what about after you connect? Do these options get >>>>> reset >>>>> to default? >>>>> >>>>> - Bootstrapping/upgrade: So far my best idea is to make the client share >>>>> it's config with the mon on startup, and the first time a given daemon >>>>> connects the mon will use that to populate it's config database. >>>>> Thereafter it will be ignored. >>>>> >>>>> - OSD startup: lots of stuff happens before we authenticate. I think >>>>> there will be a new initial step to fetch config, then do all that work, >>>>> then start up for real. And a new option to bypass mon configuration >>>>> to avoid that (and for old school folks who don't want centralized >>>>> configs... e.g. mon_config = false and everything works as before). >>>>> >>>>> Feedback welcome! >>>>> sage >>>>> -- >>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>>> the body of a message to majordomo@xxxxxxxxxxxxxxx >>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>> >>>> -- >>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>> the body of a message to majordomo@xxxxxxxxxxxxxxx >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>> the body of a message to majordomo@xxxxxxxxxxxxxxx >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@xxxxxxxxxxxxxxx >> More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
