Re: Upgrade from 17.2.5 to 17.2.6 stuck at MDS

[Thomas Widhalm <widhalmt@xxxxxxxxxxxxx>]

On 11.04.23 09:16, Xiubo Li wrote:
> On 4/11/23 03:24, Thomas Widhalm wrote:
> > Hi,
> >
> > If you remember, I hit bug https://tracker.ceph.com/issues/58489 so I 
> > was very relieved when 17.2.6 was released and started to update 
> > immediately.
> Please note, this fix is not in the v17.2.6 yet in upstream code.

Thanks for the information. I misread the information in the tracker. Do 
you have a predicted schedule for the backport? Or should I go for a 
specific pre-release? I don't want to take chances but I'm desperate 
because my production system is affected and offline for several weeks now.

Thanks,
Thomas

> Thanks
>
> - Xiubo
>
>
> > But now I'm stuck again with my broken MDS. MDS won't get into 
> > up:active without the update but the update waits for them to get into 
> > up:active state. Seems like a deadlock / chicken-egg problem to me.
> >
> > Since I'm still relatively new to Ceph, could you help me?
> >
> > What I see when watching the update status:
> >
> > {
> >     "target_image": 
> > "quay.io/ceph/ceph@sha256:1161e35e4e02cf377c93b913ce78773f8413f5a8d7c5eaee4b4773a4f9dd6635",
> >     "in_progress": true,
> >     "which": "Upgrading all daemon types on all hosts",
> >     "services_complete": [
> >         "crash",
> >         "mgr",
> >     "mon",
> >     "osd"
> >     ],
> >     "progress": "18/40 daemons upgraded",
> >     "message": "Error: UPGRADE_OFFLINE_HOST: Upgrade: Failed to 
> > connect to host ceph01 at addr (192.168.23.61)",
> >     "is_paused": false
> > }
> >
> > (The offline host was one host that broke during the upgrade. I fixed 
> > that in the meantime and the update went on.)
> >
> > And in the log:
> >
> > 2023-04-10T19:23:48.750129+0000 mgr.ceph04.qaexpv [INF] Upgrade: 
> > Waiting for mds.mds01.ceph04.hcmvae to be up:active (currently up:replay)
> > 2023-04-10T19:23:58.758141+0000 mgr.ceph04.qaexpv [WRN] Upgrade: No 
> > mds is up; continuing upgrade procedure to poke things in the right 
> > direction
> >
> >
> > Please give me a hint what I can do.
> >
> > Cheers,
> > Thomas
> >
> > _______________________________________________
> > ceph-users mailing list -- ceph-users@xxxxxxx
> > To unsubscribe send an email to ceph-users-leave@xxxxxxx

Attachment: OpenPGP_signature
Description: OpenPGP digital signature

_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx