Hi Dan, well, I was not expecting a code change any ways :) Yes, this small modification to the docs should remove potential confusion. So what you are saying is that this is not like a global default that can be specialised, its a master-override for an administrator to use in emergencies. For example, to throttle/speed-up all snaptrims without changing any of the specific config values one would set in the osd_snap_trim_sleep_xyz options. (I'm writing all of this for future duckduckgo searches.) I guess the precedence system I would expect is more or less provided by masked configs like ceph config set osd/class:hdd osd_snap_trim_sleep 1 If I follow the explanation of precedence just below https://docs.ceph.com/en/latest/rados/configuration/ceph-conf/#confsec-client, the above should take precedence over this ceph config set osd_snap_trim_sleep 10 and all osd_snap_trim_sleep_xyz values. This kind of breaks the global override effect mentioned before, but puts it closer to what I expect. Looks like both worlds can be happy. By the way, question to the experts. Just from the documentation, out of these two, which one wins on host-01 for hdds: ceph config set osd/class:hdd osd_snap_trim_sleep 1 ceph config set osd/host:host-01 osd_snap_trim_sleep 10 Now have a good snap_trim_sleep everybody. Best regards, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14 ________________________________________ From: Dan van der Ster <dvanders@xxxxxxxxx> Sent: 24 February 2022 15:34:36 To: Frank Schilder Cc: ceph-users Subject: Re: ceph fs snaptrim speed Hi Frank, Thanks for the feedback -- improving the docs is in everyone's best interest. This semantic of "override if non-zero" is quite common in the OSD. See https://github.com/ceph/ceph/blob/master/src/osd/OSD.cc#L3388-L3451 for a few examples. So it doesn't make sense to change the way this works -- this is IMO a clean way to have defaults tuned per device type, while letting a user easily override the setting without having to dive into which device type setting to change, which can easily be confusing. But we can improve the doc to make this more obvious. Does this help? https://github.com/ceph/ceph/pull/45144 Cheers, Dan On Thu, Feb 24, 2022 at 12:47 PM Frank Schilder <frans@xxxxxx> wrote: > > Hi Dan, > > thanks for the fast reply. Yes, I think the doc should be updated, because: > > > if (osd_snap_trim_sleep > 0) > > return osd_snap_trim_sleep; > > means "a value of osd_snap_trim_sleep>0 will override backend specific variants.", which is very different from the current formulation. The current formulation implies, strictly speaking, that osd_snap_trim_sleep will *always* be used and there is no point for even having the other options. > > Apart from that, I wonder if it is a good idea to have such an inconsistency in how options are applied. There is the good old software engineering paradigm of "closeness". The closer an option to its object, the higher its precedence. Like any settings in your /home override global settings and so on. More specific options should always override less specific options. But as a bare minimum, the options should at least behave the same (=parallelism, also a good old software paradigm). The current behaviour seems to violate more or less all of this. > > I would expect and also prefer that all options behave the same, that is, > > deamon specific (eg. osd.0) overrides masked (eg. class:hdd) overrides built-in specific (*_hdd) overrides daemon group (eg. ceph config osd) overrides global defaults (eg. ceph config global ...) > > Then I don't have to look at docs all the time to figure out what works how. It would all be the same. > > Thanks again! > ================= > Frank Schilder > AIT Risø Campus > Bygning 109, rum S14 > > ________________________________________ > From: Dan van der Ster <dvanders@xxxxxxxxx> > Sent: 24 February 2022 12:31:11 > To: Frank Schilder > Cc: ceph-users > Subject: Re: ceph fs snaptrim speed > > Hi Frank, > > The semantic of osd_snap_trim_sleep was copied from osd_delete_sleep. > > The general setting "osd_snap_trim_sleep" is used only to override the > _hdd _hybrid _ssd tuned values. > > Here's the code to get the effective sleep value: > > if (osd_snap_trim_sleep > 0) > return osd_snap_trim_sleep; > if (!store_is_rotational && !journal_is_rotational) > return osd_snap_trim_sleep_ssd > if (store_is_rotational && !journal_is_rotational) > return osd_snap_trim_sleep_hybrid > return osd_snap_trim_sleep_hdd > > > Do you think the doc needs to be updated? > > Cheers, Dan > > > On Thu, Feb 24, 2022 at 12:27 PM Frank Schilder <frans@xxxxxx> wrote: > > > > I seem to have the opposite problem with fs snaptrim progress as others recently. I would like to speed it up. I looked at the docs and find the description of osd_snap_trim_sleep* (https://docs.ceph.com/en/latest/rados/configuration/osd-config-ref/#confval-osd_snap_trim_sleep) counter-intuitive and confusing. > > > > For osd_snap_trim_sleep it says "This option overrides backend specific variants.", which is exactly the opposite of how other options are applied, for example, the osd_memory_target family of options. The usual way is that more specific options (back-end specific) supersede less specific options (general defaults). What is correct? Is the overall default sleep=0 for all types of OSDs, or is it 5s(=osd_snap_trim_sleep_hdd) for HDDs? > > > > If the documentation is wrong, please correct. > > > > Thanks and best regards, > > ================= > > Frank Schilder > > AIT Risø Campus > > Bygning 109, rum S14 > > _______________________________________________ > > ceph-users mailing list -- ceph-users@xxxxxxx > > To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |