Re: Ceph Upgrade 18.2.2 -> 18.2.4 fails | Fatal glibc error: CPU does not support x86-64-v2 on virtualized hosts

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Björn,

have a look at the Type "max". Thus the proxmox cluster should choose the maximum cpu type, which allows for live migration and than ceph should run with v18.2.4 images.

https://forum.proxmox.com/threads/max-cpu-type.94736/

Best regards,
Gunnar



--- Original Nachricht ---
Betreff:  Ceph Upgrade 18.2.2 -> 18.2.4 fails | Fatal glibc error: CPU does not support x86-64-v2 on virtualized hosts
Von: "Björn Lässig" <b.laessig@xxxxxxxxxxxxxx>
An: "ceph-users" <ceph-users@xxxxxxx>
Datum: 25-07-2024 10:03



Hi there,

Last week I upgraded from 17.2.7 to 18.2.2 and the cluster worked fine
except for a few bugs about IPv6.
But it is working and cluster.health is HEALTH_OK.
Hoping to fix some of these minor IPv6 issues, I tried upgrading to
18.2.4. I see 2 errors:

The first is that the upgrade fails to:

{
    "target_image": "quay.io/ceph/ceph@sha256:6ac7f923aa1d23b43248ce0ddec7e1388855ee3d00813b52c3172b0b23b37906",
    "in_progress": true,
    "which": "Upgrading all daemon types on all hosts",
    "services_complete": [],
    "progress": "0/89 daemons upgraded",
    "message": "Error: UPGRADE_FAILED_PULL: Upgrade: failed to pull target image",
    "is_paused": true
}
The error message is misleading as:

# podman images --digests
REPOSITORY                        TAG         DIGEST                                                                   IMAGE ID      CREATED        SIZE
quay.io/ceph/ceph                 <none>      sha256:6ac7f923aa1d23b43248ce0ddec7e1388855ee3d00813b52c3172b0b23b37906  2bc0b0f4375d  33 hours ago   1.25 GB
[…]

the target image was pulled without problems.
(podman pull runs without problems)

The error occures while starting the image:

Jul 25 09:16:18 cephmgr1 podman[1171476]: 2024-07-25 09:16:18.752832293 +0200 CEST m=+0.047066303 container create 4368eba845dcbc6f3cb7fe8265fc7044d91208f87c25cbf1abe6c3d0447b468e (image=quay.io/ceph/ceph@sha256:6ac7f923aa1d23b43248ce0ddec7e1388855ee3d00813b52c3172b0b23b37906, name=reverent_hypatia, org.label-schema.build-date=20240716, maintainer=Guillaume Abrioux <gabrioux@xxxxxxxxxx>, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=HEAD, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.33.8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True)
Jul 25 09:16:18 cephmgr1 systemd[1]: Started libpod-conmon-4368eba845dcbc6f3cb7fe8265fc7044d91208f87c25cbf1abe6c3d0447b468e.scope.
Jul 25 09:16:18 cephmgr1 systemd[1]: Started libpod-4368eba845dcbc6f3cb7fe8265fc7044d91208f87c25cbf1abe6c3d0447b468e.scope - libcrun container.
Jul 25 09:16:18 cephmgr1 podman[1171476]: 2024-07-25 09:16:18.820823054 +0200 CEST m=+0.115057064 container init 4368eba845dcbc6f3cb7fe8265fc7044d91208f87c25cbf1abe6c3d0447b468e (image=quay.io/ceph/ceph@sha256:6ac7f923aa1d23b43248ce0ddec7e1388855ee3d00813b52c3172b0b23b37906, name=reverent_hypatia, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.4, org.label-schema.build-date=20240716, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.33.8, GIT_BRANCH=HEAD, RELEASE=HEAD, ceph=True, maintainer=Guillaume Abrioux <gabrioux@xxxxxxxxxx>, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True)
Jul 25 09:16:18 cephmgr1 podman[1171476]: 2024-07-25 09:16:18.826093069 +0200 CEST m=+0.120327079 container start 4368eba845dcbc6f3cb7fe8265fc7044d91208f87c25cbf1abe6c3d0447b468e (image=quay.io/ceph/ceph@sha256:6ac7f923aa1d23b43248ce0ddec7e1388855ee3d00813b52c3172b0b23b37906, name=reverent_hypatia, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux <gabrioux@xxxxxxxxxx>, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=-18.2.4, RELEASE=HEAD, org.label-schema.build-date=20240716, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.33.8)
Jul 25 09:16:18 cephmgr1 podman[1171476]: 2024-07-25 09:16:18.726428757 +0200 CEST m=+0.020662767 image pull  quay.io/ceph/ceph@sha256:6ac7f923aa1d23b43248ce0ddec7e1388855ee3d00813b52c3172b0b23b37906
Jul 25 09:16:18 cephmgr1 podman[1171476]: 2024-07-25 09:16:18.82623823 +0200 CEST m=+0.120472250 container attach 4368eba845dcbc6f3cb7fe8265fc7044d91208f87c25cbf1abe6c3d0447b468e (image=quay.io/ceph/ceph@sha256:6ac7f923aa1d23b43248ce0ddec7e1388855ee3d00813b52c3172b0b23b37906, name=reverent_hypatia, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.4, RELEASE=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240716, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.33.8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux <gabrioux@xxxxxxxxxx>)
Jul 25 09:16:18 cephmgr1 reverent_hypatia[1171489]: Fatal glibc error: CPU does not support x86-64-v2
Jul 25 09:16:18 cephmgr1 systemd[1]: libpod-4368eba845dcbc6f3cb7fe8265fc7044d91208f87c25cbf1abe6c3d0447b468e.scope: Deactivated successfully.

When starting the container for 18.2.4. a glibc error occurs.
18.2.2. to 18.2.4 is a minor upgrade and should run on same
hardware but this is not hardware:
Both mgr and 3 of 5 monitor daemons run on virtual machines
on a proxmox (qemu-based virtualizaton) which has the default
CPU "kvm64". This means for the qemu options:
kvm ... -smp 1,sockets=1,cores=4,maxcpus=4 \
  -device kvm64-x86_64-cpu,id=cpu2,socket-id=0,core-id=1,thread-id=0\
  -device kvm64-x86_64-cpu,id=cpu3,socket-id=0,core-id=2,thread-id=0\
  -device kvm64-x86_64-cpu,id=cpu4,socket-id=0,core-id=3,thread-id=0

Which CPU type should I choose for my VMs for this libc?

regards
Björn Lässig
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx

Attachment: smime.p7s
Description: S/MIME cryptographic signature

_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux