On Tue, 27 Jun 2017, Kyle Bader wrote: > On Wed, Mar 8, 2017 at 6:39 AM, Sage Weil <sage@xxxxxxxxxxxx> wrote: > > On Wed, 8 Mar 2017, Dan van der Ster wrote: > >> Hi Loic, > >> > >> Did you already have a plan for how an operator would declare the > >> device class of each OSD? > >> Would this be a new --device-class option to ceph-disk prepare, which > >> would perhaps create a device-class file in the root of the OSD's xfs > >> dir? > >> Then osd crush create-or-move in ceph-osd-prestart.sh would be a > >> combination of ceph.conf's "crush location" and this per-OSD file. > > > > Hmm we haven't talked about this part yet. I see a few options... > > > > 1) explicit ceph-disk argument, recorded as a file in osd_data > > > > 2) osd can autodetect this based on the 'rotational' flag in sysfs. The > > trick here, I think, is to come up with suitable defaults. We might have > > NVMe, SATA/SAS SSDs, HDD, or a combination (with journal and data (and > > db) spread across multiple types). Perhaps those could break down into > > classes like > > > > hdd > > ssd > > nvme > > hdd+ssd-journal > > hdd+nvme-jouranl > > hdd+ssd-db+nvme-jouranl > > > > which is probably sufficient for most users. And if the admin likes they > > can override. > > > > - Then the osd adjusts device-class on startup, just like it does with the > > crush map position. (Note that this will have no real effect until the > > CRUSH rule(s) are changed to use device class.) > > > > - We'll need an 'osd crush set-device-class <osd.NNN> <class>' command. > > The only danger I see here is that if you set it to something other than > > what the OSD autodetects above, it'll get clobbered on the next OSD > > restart. Maybe the autodetection *only* sets the device class if it isn't > > already set? > > one of the classes could be "autodetect" and it could be the default, > kinda like auto negotiation on nic / switch ports? Yes! I think we already have osd_crush_update_on_start (default: yes) which controls whether we update the crush location on OSD start. We can do the same with the device class if it not already set via an option like osd_crush_update_class_on_start. I'm thinking just a simple 'hdd' and 'ssd' class... sage -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
