We have fundamental problems with the concept of cephadm and its
direction of travel. But that's a different story.
The nub of this problem is a design incompatibility with MGR and the
PyO3 package that python-cryptography relies on. It's actually unsafe as
it is, and the new package just stops you performing the unsafe
operations. So that affects all distributions and containers and
versions of ceph. Eventually the updated PyO3 will find its way into
other distributions and containers bringing things to a head.
On 02/02/2024 16:45, Brian Chow wrote:
Would migrating to a cephadm orchestrated docker/podman cluster be an
acceptable workaround?
We are running that config with reef containers on Debian 12 hosts, with a
couple of debian 12 clients successfully mounting cephfs mounts, using the
reef client packages directly on Debian.
On Fri, Feb 2, 2024, 8:21 AM Chris Palmer <chris.palmer@xxxxxxxxx> wrote:
Hi Matthew
AFAIK the upgrade from quincy/deb11 to reef/deb12 is not possible:
* The packaging problem you can work around, and a fix is pending
* You have to upgrade both the OS and Ceph in one step
* The MGR will not run under deb12 due to the PyO3 lack of support for
subinterpreters.
If you do attempt an upgrade, you will end up stuck with a partially
upgraded cluster. The MONs will be on deb12/reef and cannot be
downgraded, and the MGR will be stuck on deb11/quincy, We have a test
cluster in that state with no way forward or back.
I fear the MGR problem will spread as time goes on and PyO3 updates
occur. And it's not good that it can silently corrupt in the existing
apparently-working installations.
No-one has picked up issue 64213 that I raised yet.
I'm tempted to raise another issue for qa : the debian 12 package cannot
have been tested as it just won't work either as an upgrade or a new
install.
Regards, Chris
On 02/02/2024 14:40, Matthew Darwin wrote:
Chris,
Thanks for all the investigations you are doing here. We're on
quincy/debian11. Is there any working path at this point to
reef/debian12? Ideally I want to go in two steps. Upgrade ceph first
or upgrade debian first, then do the upgrade to the other one. Most of
our infra is already upgraded to debian 12, except ceph.
On 2024-01-29 07:27, Chris Palmer wrote:
I have logged this as https://tracker.ceph.com/issues/64213
On 16/01/2024 14:18, DERUMIER, Alexandre wrote:
Hi,
ImportError: PyO3 modules may only be initialized once per
interpreter
process
and ceph -s reports "Module 'dashboard' has failed dependency: PyO3
modules may only be initialized once per interpreter process
We have the same problem on proxmox8 (based on debian12) with ceph
quincy or reef.
It seem to be related to python version on debian12
(we have no fix for this currently)
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
