> > But, even if I gave --image flag with bootstrap the daemons created by mgr > module are using the daemon-base image, in our case its ' > docker.io/ceph/daemon-base:latest-pacific-devel'. > Which I guess is because, mgr daemon takes into consideration the > configuration parameter 'container_image', whose default value is ' > docker.io/ceph/daemon-base:latest-pacific-devel'. > What we guess is even if we provide --image flag in cephadm bootstrap, > cephadm is not updating the variable container_image with this value. > Hence, all the remaining daemons are getting created using > daemon-base image. This is not how it's supposed to work. If you provide "--image <image-name>" to bootstrap all ceph daemons deployed, including the mon/mgr deployed during bootstrap AND the daemons deployed by the cephadm mgr module afterwards should be deployed with the image provided to the "--image" parameter. You shouldn't need to set any config options or do anything extra to get that to work. If you're providing "--image" to bootstrap and this is not happening there is a serious bug (not including the fact that the bootstrap mgr/mon show the tag while others show the digest, that's purely cosmetic). If that's the case if you could post the full bootstrap output and the contents of the config file you're passing to bootstrap and maybe we can debug. I've never seen this issue before anywhere else so I have no way to recreate it (for me passing --image in bootstrap causes all ceph daemons to be deployed with that image until I explicitly specify another image through upgrade or other means). Also, regarding the non-uniform behaviour of the first mon even if created > using the same image is quite surprising. I double checked the > configuration of all mon, and could not find a major difference between > first and remaining mons. I tried to reconfigt the first mon which ended up > in the same corner. However, redeploying the specific mon with command > 'ceph orch redeploy <name> quay.io/ceph/ceph:v16.2.7, caused the first > mon also showing the same warning as rest, as it got redeployed by the mgr. Are we expecting any difference between the mon deployed by cephadm > bootstrap and mon deployed by mgr, even if we'r using the same image? > We have only the lack of warning in the first mon to state that there > might be a difference in the first mon and rest of the mons. I could maybe see some difference if you add specific config options as the mon deployed during bootstrap is deployed with basic settings. Since we can't infer config settings into the mon store until there is an existing monitor this is sort of necessary and could maybe cause some differences between that mon and others. This should be resolved by a redeploy of the mon. Can you tell me if you're setting any mon related config options in the conf you're providing to bootstrap (or if you've set any config options elsewhere). It may be that cephadm needs to actively redeploy the mon if certain options are provided in and I can look into it if I know which sorts of config options are causing the health warning. I haven't seen that health warning in my own testing (on the bootstrap mon or those deployed by the mgr module) so I'd need to know what's causing it to come about to come up with a good fix. - Adam King On Thu, Feb 3, 2022 at 11:29 AM Arun Vinod <arunvinod.tech@xxxxxxxxx> wrote: > Hi Adam, > > Thanks for reviewing the long output. > > Like you said, it makes total sense now since the first mon and mgr are > created by cephamd bootstrap and the rest of the dameons by the mgr module. > > But, even if I gave --image flag with bootstrap the daemons created by mgr > module are using the daemon-base image, in our case its ' > docker.io/ceph/daemon-base:latest-pacific-devel'. > Which I guess is because, mgr daemon takes into consideration the > configuration parameter 'container_image', whose default value is ' > docker.io/ceph/daemon-base:latest-pacific-devel'. > > What we guess is even if we provide --image flag in cephadm bootstrap, > cephadm is not updating the variable container_image with this value. > Hence, all the remaining daemons are getting created using > daemon-base image. > > Below is the value of config 'container_image' after bootstrapping with > --image flag provided. > > [root@hcictrl01 stack_orchestrator]# ceph-conf -D | grep -i > container_image > container_image = docker.io/ceph/daemon-base:latest-pacific-devel > > However, one workaround is to provide this config in the initial bootstrap > config file and present it to the cepham bootstrap using the flag --config, > which updates the image name and all the daemons are getting created with > the same image. > > Also, regarding the non-uniform behaviour of the first mon even if created > using the same image is quite surprising. I double checked the > configuration of all mon, and could not find a major difference between > first and remaining mons. I tried to reconfigt the first mon which ended up > in the same corner. However, redeploying the specific mon with command > 'ceph orch redeploy <name> quay.io/ceph/ceph:v16.2.7, caused the first > mon also showing the same warning as rest, as it got redeployed by the mgr. > > Are we expecting any difference between the mon deployed by cephadm > bootstrap and mon deployed by mgr, even if we'r using the same image? > We have only the lack of warning in the first mon to state that there > might be a difference in the first mon and rest of the mons. > > Thanks again Adam for checking this. Your insights into this will be > highly appreciated. > > Thanks and Regards, > Arun Vinod > > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |