Re: Jewel -> Luminous upgrade, package install stopped all daemons

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



This isn't a great solution, but something you could try.  If you stop all of the daemons via systemd and start them all in a screen as a manually running daemon in the foreground of each screen... I don't think that yum updating the packages can stop or start the daemons.  You could copy and paste the running command (viewable in ps) to know exactly what to run in the screens to start the daemons like this.

On Wed, Sep 13, 2017 at 6:53 PM David <dclistslinux@xxxxxxxxx> wrote:
Hi All

I did a Jewel -> Luminous upgrade on my dev cluster and it went very smoothly.

I've attempted to upgrade on a small production cluster but I've hit a snag.

After installing the ceph 12.2.0 packages with "yum install ceph" on the first node and accepting all the dependencies, I found that all the OSD daemons, the MON and the MDS running on that node were terminated. Systemd appears to have attempted to restart them all but the daemons didn't start successfully (not surprising as first stage of upgrading all mons in cluster not completed). I was able to start the MON and it's running. The OSDs are all down and I'm reluctant to attempt to start them without upgrading the other MONs in the cluster. I'm also reluctant to attempt upgrading the remaining 2 MONs without understanding what happened.

The cluster is on Jewel 10.2.5 (as was the dev cluster)
Both clusters running on CentOS 7.3

The only obvious difference I can see between the dev and production is the production has selinux running in permissive mode, the dev had it disabled.

Any advice on how to proceed at this point would be much appreciated. The cluster is currently functional, but I have 1 node out 4 with all OSDs down. I had noout set before the upgrade and I've left it set for now.

Here's the journalctl right after the packages were installed (hostname changed):

https://pastebin.com/fa6NMyjG

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux