Re: Upgarde from 16.2.1 to 16.2.2 pacific stuck

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

a couple of things.
First, is there any specific reason why you're upgrading from .1 to .2? Why not directly to .15? It seems unnecessary and you're risking upgrading to a "bad" version (I believe it was 16.2.7) if you're applying evey minor release. Or why not upgrading to Quincy or Reef directly? Second, the error message has changed (https://github.com/ceph/ceph/pull/41257) from

"could not verify host allowed virtual ips"

to

"does not belong to mon public_network".

I saw this just recently during an upgrade of a cluster I didn't deploy and it turned out to be a misconfiguration issue (mismatch between cephadm ssh-user and missing pub key). I recommend to verify the ssh connections.

ceph cephadm get-ssh-config
ceph cephadm get-user
ceph cephadm get-pub-key

It also could be just a timing issue according to Sage's statement in above PR:

Oh! I know what the problem is. 1897d1c changed the way we store the per-host network interface/network info. > On upgrade, cephadm thinks there are no networks on each host until the device refresh happens.

How long did you wait?


Zitat von Edouard FAZENDA <e.fazenda@xxxxxxx>:

Dear Ceph Community,



I am in the process of upgrading ceph pacific 16.2.1 to 16.2.2 , I have
followed the documentation :
https://docs.ceph.com/en/pacific/cephadm/upgrade/



My cluster is in Healthy state , but the upgrade is not going forward , as
on the cephadm logs I have the following :



# Ceph -W cephadm

2024-03-06T08:39:11.653447+0000 mgr.rke-sh1-1.qskoyj [INF] Upgrade: Need to
upgrade myself (mgr.rke-sh1-1.qskoyj)

2024-03-06T08:39:12.281386+0000 mgr.rke-sh1-1.qskoyj [INF] Upgrade: Updating
mgr.rke-sh1-2.lxmguj

2024-03-06T08:39:12.286096+0000 mgr.rke-sh1-1.qskoyj [INF] Deploying daemon
mgr.rke-sh1-2.lxmguj on rke-sh1-2

2024-03-06T08:39:19.347877+0000 mgr.rke-sh1-1.qskoyj [INF] Filtered out host
rke-sh1-1: could not verify host allowed virtual ips

2024-03-06T08:39:19.347989+0000 mgr.rke-sh1-1.qskoyj [INF] Filtered out host
rke-sh1-3: could not verify host allowed virtual ips

2024-03-06T08:39:19.366355+0000 mgr.rke-sh1-1.qskoyj [INF] Upgrade: Need to
upgrade myself (mgr.rke-sh1-1.qskoyj)

2024-03-06T08:39:19.965822+0000 mgr.rke-sh1-1.qskoyj [INF] Upgrade: Updating
mgr.rke-sh1-2.lxmguj

2024-03-06T08:39:19.969089+0000 mgr.rke-sh1-1.qskoyj [INF] Deploying daemon
mgr.rke-sh1-2.lxmguj on rke-sh1-2

2024-03-06T08:39:26.961455+0000 mgr.rke-sh1-1.qskoyj [INF] Filtered out host
rke-sh1-1: could not verify host allowed virtual ips

2024-03-06T08:39:26.961502+0000 mgr.rke-sh1-1.qskoyj [INF] Filtered out host
rke-sh1-3: could not verify host allowed virtual ips

2024-03-06T08:39:26.973897+0000 mgr.rke-sh1-1.qskoyj [INF] Upgrade: Need to
upgrade myself (mgr.rke-sh1-1.qskoyj)

2024-03-06T08:39:27.623773+0000 mgr.rke-sh1-1.qskoyj [INF] Upgrade: Updating
mgr.rke-sh1-2.lxmguj

2024-03-06T08:39:27.628115+0000 mgr.rke-sh1-1.qskoyj [INF] Deploying daemon
mgr.rke-sh1-2.lxmguj on rke-sh1-2



My public_network is set :



root@rke-sh1-1:~# ceph config dump  | grep public_network

  mon                                          advanced  public_network
10.10.71.0/24

                                                                     *

Do you have an idea why I have the following error :



Filtered out host: could not verify host allowed virtual ips





Current state of the upgrade :



# ceph orch upgrade status

{

    "target_image":
"docker.io/ceph/ceph@sha256:8cdd8c7dfc7be5865255f0d59c048a1fb8d1335f6923996e
2c2d9439499b5cf2",

    "in_progress": true,

    "services_complete": [],

    "progress": "0/35 ceph daemons upgraded",

    "message": "Currently upgrading mgr daemons"

}



  progress:

    Upgrade to 16.2.2 (24m)

      [............................]



Thanks for the help.



Best Regards,



Edouard FAZENDA

Technical Support







Chemin du Curé-Desclouds 2, CH-1226 THONEX  +41 (0)22 869 04 40



 <https://www.csti.ch/> www.csti.ch


_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux