Carlos,
Carlos Mogas da Silva wrote:
: >From what I can gather, this will not be smooth at all, since I can't make an inplace upgrade of the
: OS first and then Ceph and neither other way around. So the idea is to create a total new Ceph
: cluster from scratch and migrate the data from one to another. The question is, how do I do this
: last step?
I've just finished the in-place migration from C7 to C8Stream myself.
What I did was:
- upgrade the cluster to C7 + Octopus, because Octopus packages are available
both for C7 and C8.
- upgrade the OS on the mons one by one, preserving the
/var/lib/ceph/mon/ceph-* directories (backup before and restore
afterwards is OK). My mon hosts also run an instance of mgr and rgw,
so start also these. I did this at least a month ago, so I don't exactly
remember whether I did some more magic somewhere :-). But the cluster has
been running happily for at least a month with C8Stream+Octopus mons
and C7+Octopus OSDs.
- ceph osd set noout
- upgrade the osd hosts one by one:
- create a kickstart.cfg file which deletes all the non-ceph partitions,
creates new ones, and installs C8Stream there.
I have OSD data on whole disks without partitions, so the installer does
not have a chance to overwrite them accidentally - it would fail when
trying to delete the old system partitions first, even if the order of
the disks gets changed somehow. Something like this:
ignoredisk --only-use=nvme0n1,nvme1n1
clearpart --list=nvme0n1p1,nvme0n1p2,nvme0n1p3,nvme1n1p1,nvme1n1p2,nvme1n1p3
- I set up Kickstart.cfg as a minimal install of C8Stream with ansible SSH
public key added to /root/.ssh/authorized_keys in %post (remember
to restorecon -R /root/.ssh in %post as well).
- I booted the installer using PXE+UEFI with GRUB, so that I can include the
ks=.../kickstart.cfg option there and don't have to enter it manually.
Just don't make the grub entry with disk-overwriting kickstart.cfg the
default one :-)
- In dhcpd.conf, I have
next-server my.tftp.server;
filename "c8stream/grubx64.efi";
- I've put grubx64.efi, vmlinuz, grub.cfg and initrd.img to
my.tftp.server:/tftpboot
- the grub.cfg entries are:
set timeout=120
default skip
menuentry 'Exit GRUB (and try next boot option)' --id skip {
echo Bye :]
exit
}
menuentry 'CentOS 8 Stream Kickstart (REWRITES DISKS!!!)' {
linuxefi /c8stream/vmlinuz ip=dhcp inst.ks=http://my.http.server/kicksart.cfg
initrdefi /c8stream/initrd.img
}
- Then I used ansible to configure the rest (Ceph repository, packages,
and some other tools we need - snmpd, ...).
- Make sure to have /var/lib/ceph/bootstrap-osd/ceph.keyring installed
with the correct bootstrap-osd key.
- after the osd host is up and running, re-attach the OSD volumes with
"ceph-volume lvm activate --all"
- after "ceph -s" reports no other problem than the noout flag being set,
continue with the next OSD host.
- ceph osd unset noout
- optionally, upgrade to Pacific (I did not do that part yet)
For me, it still was a lot of manual work, because I also did a BIOS upgrade
on each OSD host, so I had to reconfigure the BIOS NVRAM as well. I guess
it took me about 30 to 60 minutes per host.
Hope this helps,
-Yenya
--
| Jan "Yenya" Kasprzak <kas at {fi.muni.cz - work | yenya.net - private}> |
| http://www.fi.muni.cz/~kas/ GPG: 4096R/A45477D5 |
We all agree on the necessity of compromise. We just can't agree on
when it's necessary to compromise. --Larry Wall
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
