Re: Rolling upgrade model to new OS

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Drew,
I performed the upgrade from Nautilus  (bare-metal deployment) -> Octopus
(podman containerization) and RHEL-7 -> RHEL-8.
Everything was done in-place. My sequence was:
ceph osd noout/norebalance
shutdown/disable running services
perform full OS upgrade
install necessary software like podman
start ceph monitor in podman
ensure it is joining the rest of cluster
ensure the monitor is stable
deploy osds with the same ids I had previously
ensure everything is stable
deploy additional daemons (if needed)
check cluster's overall health state
if everything is fine, proceed to the next node

There wasn't any major issue related to the upgrade, however I hit some
minors. E.g. Sometimes for some reason monitors didn't join the cluster. or
osds started flapping, but all problems I saw - were fixable in the
observable period of time.
My personal opinion - if you can use docker, then use docker and test the
ceph images before you deploy it into production. I used to use our staging
environment.
Very likely, you should make several changes in systemd unit files and/or
in the docker/podman command. Don't forget to mount all directories ceph is
using, like /var/lib/ceph, etc.
To troubleshoot the first start, I started the container without -d, to
monitor the container's output in real time, then when I had all mistakes
fixed, I restarted it using the systemd unit.
Unit files you can grab on ceph-ansible github.

Good luck!
Vladimir

On Fri, Jun 4, 2021 at 8:55 AM Drew Weaver <drew.weaver@xxxxxxxxxx> wrote:

> Hello,
>
> I need to upgrade the OS that our Ceph cluster is running on to support
> new versions of Ceph.
>
> Has anyone devised a model for how you handle this?
>
> Do you just:
>
> Install some new nodes with the new OS
> Install the old version of Ceph on the new nodes
> Add those nodes/osds to the cluster
> Remove the old nodes
> Upgrade Ceph on the new nodes
>
> Are there any specific OS that Ceph has said that will have longer future
> version support? Would like to only touch the OS every 3-4 years if
> possible.
>
> Thanks,
> -Drew
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx
> To unsubscribe send an email to ceph-users-leave@xxxxxxx
>
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux