Re: cephadm picks development/latest tagged image for daemon-base (docker.io/ceph/daemon-base:latest-pacific-devel)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Adam,

Thanks for replying.

I have tried the "ceph orch upgrade start <image-name>" as a workaround and
it works as expected. All the daemons are recreated with the stable version
of the image. However, it still requires an initial fetch of the
latest-pacific-devel image and creation of daemons with that.

Also,  cephadm bootstrap was not allowing --image option, so in order to
specify the image name I have updated the value of DEFAULT_IMAGE  in the
cephadm script  (/usr/bin/cephadm) as follows

DEFAULT_IMAGE = 'quay.io/ceph/ceph:v16.2.7

I have attached two logs along with this email, first one is the cephadm
log from the bootstrap node and the second one is from the second host
after adding it to the cluster.

bootstrap command  used:
*sudo cephadm bootstrap --skip-monitoring-stack --mon-ip *.*.*.*
--cluster-network *.*.*.*/24 --ssh-user ceph_user --ssh-private-key
/home/ceph_user/.ssh/id_rsa --ssh-public-key
/home/ceph_user/.ssh/id_rsa.pub --config
/home/ceph_user/ceph_bootstrap/ceph.conf  --no-minimize-config*

containers in bootstrap host:
[root@hcictrl01 ~]# podman ps -a --format "{{.Image}} {{.Command}}
{{.Names}}"
quay.io/ceph/ceph:v16.2.7 -n mon.hcictrl01 ...
ceph-6eaf15d8-8332-11ec-b820-0015171590ba-mon-hcictrl01
quay.io/ceph/ceph:v16.2.7 -n mgr.hcictrl01....
ceph-6eaf15d8-8332-11ec-b820-0015171590ba-mgr-hcictrl01-vmdxbg
docker.io/ceph/daemon-base:latest-pacific-devel -n client.crash.h...
ceph-6eaf15d8-8332-11ec-b820-0015171590ba-crash-hcictrl01

host add command used:
*ceph orch host add hcictrl02 *.*.*.* --labels _admin*

containers in second host:
[root@hcictrl02 ~]# podman ps -a --format "{{.Image}} {{.Command}}
{{.Names}}"
docker.io/ceph/daemon-base:latest-pacific-devel -n client.crash.h...
ceph-6eaf15d8-8332-11ec-b820-0015171590ba-crash-hcictrl02
docker.io/ceph/daemon-base:latest-pacific-devel -n mgr.hcictrl02....
ceph-6eaf15d8-8332-11ec-b820-0015171590ba-mgr-hcictrl02-xfbcwn
docker.io/ceph/daemon-base:latest-pacific-devel -n mon.hcictrl02 ...
ceph-6eaf15d8-8332-11ec-b820-0015171590ba-mon-hcictrl02


However, does the following line have anything to do with this behaviour of
cephadm?

https://github.com/ceph/ceph/blob/v16.2.7/src/common/options.cc#L459

[root@hcictrl01 ~]# ceph-conf -D | grep -i container_image

container_image = docker.io/ceph/daemon-base:latest-pacific-devel


Thanks and Regards,
Arun Vinod



On Mon, 31 Jan 2022 at 22:25, Adam King <adking@xxxxxxxxxx> wrote:

> Hi Arun,
>
> Not sure exactly how things got this way. When you provide "--image
> <image-name>" when bootstrapping that should set the image to be used for
> all ceph containers. I've never seen just the bootstrap mgr/mon get a
> totally different image. Would be interesting to maybe see the full
> bootstrap output here as this issue is new to me.
>
> As for resolving the issue, you should be able to use the upgrade
> procedure to get all the containers on the right image. Just "ceph orch
> upgrade start <image-name>" then just keep checking "ceph orch upgrade
> status" until it no longer says it's in progress. That should get all the
> ceph daemons on whatever image it is you specify in the upgrade start
> command and cause future ceph daemons to be deployed with that image as
> well.
>
> - Adam King
>
> On Mon, Jan 31, 2022 at 10:08 AM Arun Vinod <arunvinod.tech@xxxxxxxxx>
> wrote:
>
>> Hi All,
>>
>> How can change the default behaviour of cepham to use stable container
>> images instead of default latest/devel images.
>>
>> By default when we try to bootstrap a cluster and add two additional hosts
>> after bootstrap finished, dameons are created on two container images.
>> Which are, *quay.io/ceph/ceph:v16 <http://quay.io/ceph/ceph:v16>* and
>> *docker.io/ceph/daemon-base:latest-pacific-devel
>> <http://docker.io/ceph/daemon-base:latest-pacific-devel>*.
>>
>> The ceph:v16 image from quay.io is stable but the second image from
>> docker.io is not , due to the latest/devel tags which are basically
>> untested images according to ceph.
>>
>> How can we specify cephadm to use a stable version of image for
>> *daemon-base.*
>>
>> The following is the final list of containers created on the cluster after
>> bootstrap is finished and another host added. Here the mon, mgr daemons on
>> both hosts are running on different container image. Most importantly,
>> most
>> of the containers are running on latest-pacific-devel containers, which is
>> not apt for a production cluster.
>>
>> In bootstrap node:
>> CONTAINER ID  IMAGE                                            COMMAND
>>           CREATED      STATUS          PORTS       NAMES
>> afeb6f92deb2  quay.io/ceph/ceph:v16.2.7                        -n
>> mon.hcictrl01 ...  4 hours ago  Up 4 hours ago
>>  ceph-e8200504-8287-11ec-a14f-0015171590ba-mon-hcictrl01
>> c43d48766a08  quay.io/ceph/ceph:v16.2.7                        -n
>> mgr.hcictrl01....  4 hours ago  Up 4 hours ago
>>  ceph-e8200504-8287-11ec-a14f-0015171590ba-mgr-hcictrl01-rmosyh
>> d70cea0fd561  docker.io/ceph/daemon-base:latest-pacific-devel  -n
>> client.crash.h...  4 hours ago  Up 4 hours ago
>>  ceph-e8200504-8287-11ec-a14f-0015171590ba-crash-hcictrl01
>>
>> In rest of the nodes:
>> CONTAINER ID  IMAGE                                            COMMAND
>>           CREATED        STATUS            PORTS       NAMES
>> d816ee470753  docker.io/ceph/daemon-base:latest-pacific-devel  -n
>> client.crash.h...  6 minutes ago  Up 6 minutes ago
>>  ceph-e8200504-8287-11ec-a14f-0015171590ba-crash-hcictrl02
>> dde4646f4819  docker.io/ceph/daemon-base:latest-pacific-devel  -n
>> mgr.hcictrl02....  6 minutes ago  Up 6 minutes ago
>>  ceph-e8200504-8287-11ec-a14f-0015171590ba-mgr-hcictrl02-hfhapx
>> 12191a039525  docker.io/ceph/daemon-base:latest-pacific-devel  -n
>> mon.hcictrl02 ...  6 minutes ago  Up 6 minutes ago
>>  ceph-e8200504-8287-11ec-a14f-0015171590ba-mon-hcictrl02
>>
>> Can someone help to explain how cepham chooses this default image or any
>> workaround to choose a specific image instead of devel images.
>>
>> Thanks in advance.
>> _______________________________________________
>> ceph-users mailing list -- ceph-users@xxxxxxx
>> To unsubscribe send an email to ceph-users-leave@xxxxxxx
>>
>>
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux