The idea of making an automatic configuration is tied with the concept of opiniating what should be the kind of devices to associate.
You spoke about using a ratio between SSDs and HDDs to get a good setup.
What should be the behavior of the tool if the ratio :
- cannot be reach (not enough HDDs for 1 SSD) ?
- is exceeded (if we have 1 more HDD than expected, shall it be included or left away alone ?)
If we have several SSDs free, which one should be used ?
If we have multiple HDDs types (10/15K/7.2K) how be sure they are used in the same 'auto' setup ?
If we have 1 SSD and 1 NVMe, which one is preferred ?
What if there is some devices that should not be used by ceph-volume ? Does it imply using the manual mode ?
When the user is providing a list of devices, do we agree that they have to be checked against the "rejecting" rules to avoid using a wrong device ?
----- Mail original -----
De: "Alfredo Deza" <adeza@xxxxxxxxxx>
À: "ceph-devel" <ceph-devel@xxxxxxxxxxxxxxx>
Envoyé: Mardi 19 Juin 2018 21:35:02
Objet: ceph-volume and automatic OSD provisioning
One of the top questions for ceph-volume has been "why this doesn't create
partitions like ceph-disk does?". Although we have initially focused on LVM,
the same question is true (except for LVs instead of partitions). Now
that ceph-volume is
stabilizing, we can expand on a more user-friendly approach.
We are planning on creating an interface to size devices automatically based on
some simple criteria. There are three distinct use cases that we are going to
support, that should allow easy OSD provisioning with defaults, to more
esoteric use cases with third-party systems (like rook, ceph-ansible, seasalt,
etc...)
This is being implemented as a separate sub-command to avoid pilling up the
complexity on the existing `lvm` one, and reflect the automation behind it.
Here are some examples on how the API is being designed, for fully automatic
configuration, semi-automatic (allows input), and manual via a config
management system:
Automatic (no configuration or options required):
-------------------------------------------------
Single device type:
$ ceph-volume auto
Use --yes to run
Detected devices:
[rotational] /dev/sda 1TB
[rotational] /dev/sdb 1TB
[rotational] /dev/sdc 1TB
Expected Bluestore OSDs:
data: /dev/sda (100%)
data: /dev/sdb (100%)
data: /dev/sdc (100%)
This scenario will detect a single type of unused device (rotational)
so the bluestore
OSD will be created on each without block.db or block.wal
Mixed devices:
$ ceph-volume auto
Use --yes to run
Detected devices:
[rotational] /dev/sda 1TB
[rotational] /dev/sdb 1TB
[rotational] /dev/sdc 1TB
[solid ] /dev/sdd 500GB
Expected Bluestore OSDs:
data: /dev/sda (100%), block.db: /dev/sdd (33%)
data: /dev/sdb (100%), block.db: /dev/sdd (33%)
data: /dev/sdc (100%), block.db: /dev/sdd (33%)
This scenario will detect the unused devices in the system and understand that
there is a mix of solid and rotational devices, will place block on the
rotational ones, and will split the ssd in as many rotational devices found (3
in this case).
Semi configurable outcome, with input:
--------------------------------------
A user might not want to consume the devices that were automatically detected
in the system as free, so the interface will allow to pass these devices
directly as input.
$ ceph-volume auto /dev/sda /dev/sdb /dev/sdc
Device information:
[rotational] /dev/sda 1TB
[rotational] /dev/sdb 1TB
[rotational] /dev/sdc 1TB
Expected Bluestore OSDs:
data: /dev/sda (100%), block.db: /dev/sdd (33%)
data: /dev/sdb (100%), block.db: /dev/sdd (33%)
data: /dev/sdc (100%), block.db: /dev/sdd (33%)
Please hit Enter to continue, or Ctrl-C to cancel
Similarly, for mixed devices:
$ ceph-volume auto /dev/sda /dev/sdb /dev/sdc /dev/sdd
Use --yes to run
Device information:
[rotational] /dev/sda 1TB
[rotational] /dev/sdb 1TB
[rotational] /dev/sdc 1TB
[solid ] /dev/sdd 500GB
Expected Bluestore OSDs:
data: /dev/sda (100%), block.db: /dev/sdd (33%)
data: /dev/sdb (100%), block.db: /dev/sdd (33%)
data: /dev/sdc (100%), block.db: /dev/sdd (33%)
Please hit Enter to continue, or Ctrl-C to cancel
Fully Manual (config management systems):
-----------------------------------------
A JSON file or a blob as a positional arugment would allow fine tunning other
specifics, like using 2 OSDs per NVMe device, determine an exact size for
a block.db or even a block.wal LV.
$ ceph-volume auto /etc/ceph/custom_osd_provisioning.json
Or:
$ ceph-volume auto "{ ... }"
Here the API is still undefined as of now, but the idea is to expand on more
complex setups that can be better managed by configuration management systems
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
