Ooooo thanks John, will give it a try and report back ________________________________ From: John Mulligan <phlogistonjohn@xxxxxxxxxxxxx> Sent: 26 March 2024 14:24 To: ceph-users@xxxxxxx <ceph-users@xxxxxxx> Cc: Iain Stott <Iain.Stott@xxxxxxx> Subject: Re: Cephadm on mixed architecture hosts CAUTION: This email originates from outside THG On Tuesday, March 26, 2024 7:22:18 AM EDT Iain Stott wrote: > Hi, > > We are trying to deploy Ceph Reef 18.2.1 using cephadm on mixed architecture > hosts using x86_64 for the mons and aarch64 for the OSDs. > > During deployment we use the following config for the bootstrap process, > where $REPOSITORY is our docker repo. > > [global] > container_image = $REPOSITORY/ceph/ceph:v18.2.1 > [mgr] > mgr/cephadm/container_image_base = $REPOSITORY/ceph/ceph:v18.2.1 > mgr/cephadm/container_image_prometheus = $REPOSITORY/ceph/prometheus:v2.33.4 > mgr/cephadm/container_image_node_exporter = > $REPOSITORY/ceph/node-exporter:v1.3.1 mgr/cephadm/container_image_grafana = > $REPOSITORY/ceph/ceph-grafana:8.3.5 > mgr/cephadm/container_image_alertmanager = > $REPOSITORY/ceph/alertmanager:v0.23.0 [osd] > container_image = $REPOSITORY/ceph/ceph:v18.2.1 > Once the bootstrap process is complete, if we do a ceph config dump, the > container image for global and osd changes from version tag to sha > reference, meaning that when deploying the containers on the OSDs they try > using the amd64 container image and not the aarch64 image and fail > deployment. > > Is there a config setting we are missing or a workaround for this? > Try: `ceph config set mgr mgr/cephadm/use_repo_digest false` This comes up often enough that we should document it. I don't think that option is documented right now because I searched for the option with google and all that came up were other tracker issues and older mailing-list posts. In the longer term we may want to make cephadm arch-aware (volunteers welcome :-) ). _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |