My €0.02 for what it's worth(less). I've been doing RBD-based VMs under libvirt with no problem. In that particular case, the ceph RBD base images are being overlaid cloud- style with a an instance-specific qcow2 image and the RBD is just part of my storage pools. For a physical machine, I'd probably be content to use the Ceph NFS share for the OS and PXE boot itself. I could always mount a ceph filesystem once the OS was up if I wanted to. If I'm not mistaken, getting ceph to boot directly would require a custom ramdisk with ceph driver installed, whereas an NFS-supporting ramdisk is pretty much off-the-shelf. Since ceph-ganesha NFS seems pretty reliable and performant, I'd be happy with that. Tim On Thu, 2024-01-04 at 17:16 +0100, Nico Schottelius wrote: > > We also had a look at this a few years ago when we flashed almost all > servers to boot iPXE from the nic directly. The idea was as follows: > > iPXE -> http -> get kernel + initramfs -> mount rootfs > > Our idea was to use RBD as a disk for the server, but to the last of > my > knowledge, there is no support in the linux kernel for something like > > linux root=ceph://mon1,mon2,mon3,key=keytouse/rbd/pool-x/image-y > > Which would be needed for a ceph based rootfs. > > It would be amazing to have that though, a decentralised storage > system > that is able to be booted from. Above description would actually be > more > suitable to the requested cephfs booting, as NFS root also often > splits > the kernel+initramfs. > > The RBD case would potentially require boot loader support, if we > assume > that the RBD image contains something like grub - so a tiny bit more > complicated. > > BR and wishing you all the best for this approach, > > Nico > > > Alex Gorbachev <ag@xxxxxxxxxxxxxxxxxxx> writes: > > > A while ago I had done this with iSCSI, using iBFT, the advantage > > being the > > NIC itself supports such boot, the disadvantage is you need to use > > iSCSI. > > > > I recall PXE can bring an image over TFTP, so assuming you want to > > load > > that image into an RBD device for persistence? Looks like this was > > discussed here: > > https://www.spinics.net/lists/ceph-users/msg66370.html > > > -- > Sustainable and modern Infrastructures by ungleich.ch > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |