On Tue, Aug 6, 2019 at 11:11 AM Paul Emmerich <paul.emmerich@xxxxxxxx> wrote:
On Tue, Aug 6, 2019 at 7:45 PM Robert LeBlanc <robert@xxxxxxxxxxxxx> wrote:
> We have a 12.2.8 luminous cluster with all NVMe and we want to take some of the NVMe OSDs and allocate them strictly to metadata pools (we have a problem with filling up this cluster and causing lingering metadata problems, and this will guarantee space for metadata operations).
Depending on the workload and metadata size: this might be a bad idea
as it reduces parallelism.
We will still have 13 OSDs, it will be overkill for space for metadata, but since Ceph lacks a reserve space feature, we don't have many options. This cluster is so fast that it can fill up in the blink of an eye.
> In the past, we have done this the old-school way of creating a separate root, but I wanted to see if we could leverage the device class function instead.
>
> Right now all our devices show as ssd rather than nvme, but that is the only class in this cluster. None of the device classes were manually set, so is there a reason they were not detected as nvme?
Ceph only distinguishes rotational vs. non-rotational for device
classes and device-specific configuration options
I read https://ceph.com/community/new-luminous-crush-device-classes/ and it mentions that Ceph automatically classifies into hdd, ssd, and nvme. Hence the question.
> Is it possible to add a new device class like 'metadata'?
yes, it's just a string, you can use whatever you want. For example,
we run a few setups that distinguish 2.5" and 3.5" HDDs.
>
> If I set the device class manually, will it be overwritten when the OSD boots up?
not sure about this, we run all of our setups with auto-updating on
start disabled
Do you know if 'osd crush location hook' can specify the device class?
> Is what I'm trying to accomplish better done by the old-school separate root and the osd_crush_location_hook (potentially using a file with a list of partition UUIDs that should be in the metadata pool).?
device classes are the way to go here
Thanks!
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
