On 12/01/2019 15:07, Brian Topping wrote:
I’m a little nervous that BlueStore assumes it owns the partition table and will not be happy that a couple of primary partitions have been used. Will this be a problem?
You should look into using ceph-volume in LVM mode. This will allow you to create an OSD out of any arbitrary LVM logical volume, and it doesn't care about other volumes on the same PV/VG. I'm running BlueStore OSDs sharing PVs with some non-Ceph stuff without any issues. It's the easiest way for OSDs to coexist with other stuff right now.
So, for example, you could have /boot on a partition, an LVM PV on another partition, containing an LV for / and an LV for your OSD. Or you could just use a partition for / (including /boot) and just have another partition for a PV wholly occupied by a single OSD LV. How you set up everything around LVM is up to you, ceph-volume just wants a logical volume to own (and uses LVM metadata to store its stuff, so it doesn't require a separate filesystem for metadata, just the main BlueStore device).
I also have two clusters using ceph-disk with a rootfs RAID1 across OSD data drives, with extra partitions; at least with GPT this works without any problems, but set-up might be finicky. For our deployment I ended up rewriting what ceph-disk does in my own script (it's not that complicated, just create a few partitions with the right GUIDs and write some files to the OSD filesystem root). So the OSDs get set up with some custom code, but then normal usage just uses ceph-disk (it certainly doesn't care about extra partitions once everything is set up). This was formerly FileStore and now BlueStore, but it's a legacy setup. I expect to move this over to ceph-volume at some point.
-- Hector Martin (hector@xxxxxxxxxxxxxx) Public Key: https://marcan.st/marcan.asc _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
