Don't know if it helps, but we have also experienced something similar
with osd images. We changed the image tag from version to sha and it did
not happen again.
___________________________________
ceph ambassador DACH
ceph consultant since 2012
Clyso GmbH - Premier Ceph Foundation Member
https://www.clyso.com/
Am 15.05.23 um 14:50 schrieb Adam King:
I think with the `config set` commands there is logic to notify the
relevant mgr modules and update their values. That might not exist with
`config rm`, so it's still using the last set value. Looks like a real bug.
Curious what happens if the mgr restarts after the `config rm`. Whether it
goes back to the default image in that case or not. Might take a look later.
On Mon, May 15, 2023 at 7:37 AM Daniel Krambrock <
krambrock@xxxxxxxxxxxxxxxxxx> wrote:
Hello.
I think i found a bug in cephadm/ceph orch:
Redeploying a container image (tested with alertmanager) after removing
a custom `mgr/cephadm/container_image_alertmanager` value, deploys the
previous container image and not the default container image.
I'm running `cephadm` from ubuntu 22.04 pkg 17.2.5-0ubuntu0.22.04.3 and
`ceph` version 17.2.6.
Here is an example. Node clrz20-08 is the node altermanager is running
on, clrz20-01 the node I'm controlling ceph from:
* Get alertmanager version
```
root@clrz20-08:~# cephadm ls | jq '.[] | select(.service_name ==
"alertmanager")| .container_image_name'
"quay.io/prometheus/alertmanager:v0.23.0"
```
* Set alertmanager image
```
root@clrz20-01:~# ceph config set mgr
mgr/cephadm/container_image_alertmanager quay.io/prometheus/alertmanager
root@clrz20-01:~# ceph config get mgr
mgr/cephadm/container_image_alertmanager
quay.io/prometheus/alertmanager
```
* redeploy altermanager
```
root@clrz20-01:~# ceph orch redeploy alertmanager
Scheduled to redeploy alertmanager.clrz20-08 on host 'clrz20-08'
```
* Get alertmanager version
```
root@clrz20-08:~# cephadm ls | jq '.[] | select(.service_name ==
"alertmanager")| .container_image_name'
"quay.io/prometheus/alertmanager:latest"
```
* Remove alertmanager image setting, revert to default:
```
root@clrz20-01:~# ceph config rm mgr
mgr/cephadm/container_image_alertmanager
root@clrz20-01:~# ceph config get mgr
mgr/cephadm/container_image_alertmanager
quay.io/prometheus/alertmanager:v0.23.0
```
* redeploy altermanager
```
root@clrz20-01:~# ceph orch redeploy alertmanager
Scheduled to redeploy alertmanager.clrz20-08 on host 'clrz20-08'
```
* Get alertmanager version
```
root@clrz20-08:~# cephadm ls | jq '.[] | select(.service_name ==
"alertmanager")| .container_image_name'
"quay.io/prometheus/alertmanager:latest"
```
-> `mgr/cephadm/container_image_alertmanager` is set to
`quay.io/prometheus/alertmanager:v0.23.0`
<http://quay.io/prometheus/alertmanager:v0.23.0>, but redeploy uses
`quay.io/prometheus/alertmanager:latest`
<http://quay.io/prometheus/alertmanager:latest>. This looks like a bug.
* Set alertmanager image explicitly to the default value
```
root@clrz20-01:~# ceph config set mgr
mgr/cephadm/container_image_alertmanager
quay.io/prometheus/alertmanager:v0.23.0
root@clrz20-01:~# ceph config get mgr
mgr/cephadm/container_image_alertmanager
quay.io/prometheus/alertmanager:v0.23.0
```
* redeploy altermanager
```
root@clrz20-01:~# ceph orch redeploy alertmanager
Scheduled to redeploy alertmanager.clrz20-08 on host 'clrz20-08'
```
* Get alertmanager version
```
root@clrz20-08:~# cephadm ls | jq '.[] | select(.service_name ==
"alertmanager")| .container_image_name'
"quay.io/prometheus/alertmanager:v0.23.0"
```
-> Setting `mgr/cephadm/container_image_alertmanager` to the default
setting fixes the issue.
Bests,
Daniel
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
