Re: cephadm picks development/latest tagged image for daemon-base (docker.io/ceph/daemon-base:latest-pacific-devel)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



>
> But, even if I gave --image flag with bootstrap the daemons created by mgr
> module are using the daemon-base image, in our case its '
> docker.io/ceph/daemon-base:latest-pacific-devel'.
> Which I guess is because, mgr daemon takes into consideration the
> configuration parameter 'container_image', whose default value is '
> docker.io/ceph/daemon-base:latest-pacific-devel'.
> What we guess is even if we provide --image flag in cephadm bootstrap,
> cephadm is not updating the variable container_image with this value.
> Hence, all the remaining daemons are getting created using
> daemon-base image.


This is not how it's supposed to work. If you provide "--image
<image-name>" to bootstrap all ceph daemons deployed, including the mon/mgr
deployed during bootstrap AND the daemons deployed by the cephadm mgr
module afterwards should be deployed with the image provided to the
"--image" parameter. You shouldn't need to set any config options or do
anything extra to get that to work. If you're providing "--image" to
bootstrap and this is not happening there is a serious bug (not including
the fact that the bootstrap mgr/mon show the tag while others show the
digest, that's purely cosmetic). If that's the case if you could post the
full bootstrap output and the contents of the config file you're passing to
bootstrap and maybe we can debug. I've never seen this issue before
anywhere else so I have no way to recreate it (for me passing --image in
bootstrap causes all ceph daemons to be deployed with that image until I
explicitly specify another image through upgrade or other means).

Also, regarding the non-uniform behaviour of the first mon even if created
> using the same image is quite surprising. I double checked the
> configuration of all mon, and could not find a major difference between
> first and remaining mons. I tried to reconfigt the first mon which ended up
> in the same corner. However, redeploying the specific mon with command
> 'ceph orch redeploy <name> quay.io/ceph/ceph:v16.2.7, caused the first
> mon also showing the same warning as rest, as it got redeployed by the mgr.


Are we expecting any difference between the mon deployed by cephadm
> bootstrap and mon deployed by mgr, even if we'r using the same image?
> We have only the lack of warning in the first mon to state that there
> might be a difference in the first mon and rest of the mons.


I could maybe see some difference if you add specific config options as the
mon deployed during bootstrap is deployed with basic settings. Since we
can't infer config settings into the mon store until there is an existing
monitor this is sort of necessary and could maybe cause some differences
between that mon and others. This should be resolved by a redeploy of the
mon. Can you tell me if you're setting any mon related config options in
the conf you're providing to bootstrap (or if you've set any config options
elsewhere). It may be that cephadm needs to actively redeploy the mon if
certain options are provided in and I can look into it if I know which
sorts of config options are causing the health warning. I haven't seen that
health warning in my own testing (on the bootstrap mon or those deployed by
the mgr module) so I'd need to know what's causing it to come about to come
up with a good fix.


- Adam King

On Thu, Feb 3, 2022 at 11:29 AM Arun Vinod <arunvinod.tech@xxxxxxxxx> wrote:

> Hi Adam,
>
> Thanks for reviewing the long output.
>
> Like you said, it makes total sense now since the first mon and mgr are
> created by cephamd bootstrap and the rest of the dameons by the mgr module.
>
> But, even if I gave --image flag with bootstrap the daemons created by mgr
> module are using the daemon-base image, in our case its '
> docker.io/ceph/daemon-base:latest-pacific-devel'.
> Which I guess is because, mgr daemon takes into consideration the
> configuration parameter 'container_image', whose default value is '
> docker.io/ceph/daemon-base:latest-pacific-devel'.
>
> What we guess is even if we provide --image flag in cephadm bootstrap,
> cephadm is not updating the variable container_image with this value.
> Hence, all the remaining daemons are getting created using
> daemon-base image.
>
> Below is the value of config 'container_image' after bootstrapping with
> --image flag provided.
>
> [root@hcictrl01 stack_orchestrator]# ceph-conf -D | grep -i
> container_image
> container_image = docker.io/ceph/daemon-base:latest-pacific-devel
>
> However, one workaround is to provide this config in the initial bootstrap
> config file and present it to the cepham bootstrap using the flag --config,
> which updates the image name and all the daemons are getting created with
> the same image.
>
> Also, regarding the non-uniform behaviour of the first mon even if created
> using the same image is quite surprising. I double checked the
> configuration of all mon, and could not find a major difference between
> first and remaining mons. I tried to reconfigt the first mon which ended up
> in the same corner. However, redeploying the specific mon with command
> 'ceph orch redeploy <name> quay.io/ceph/ceph:v16.2.7, caused the first
> mon also showing the same warning as rest, as it got redeployed by the mgr.
>
> Are we expecting any difference between the mon deployed by cephadm
> bootstrap and mon deployed by mgr, even if we'r using the same image?
> We have only the lack of warning in the first mon to state that there
> might be a difference in the first mon and rest of the mons.
>
> Thanks again Adam for checking this. Your insights into this will be
> highly appreciated.
>
> Thanks and Regards,
> Arun Vinod
>
>
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux