Hi Adam, Thanks for clarifying it. The local registry suggestion works fine for us. We set it up and it works like a charm. Thanks and Regards, Arun Vinod On Fri, 4 Feb 2022 at 22:17, Adam King <adking@xxxxxxxxxx> wrote: > Hi Arun, > > The --skip-pull flag only stops cephadm bootstrap itself from trying to > pull the image. It doesn't affect whether the cephadm mgr module will try > to do so (ultimately even if we don't explicitly try to pull the image, it > will get pulled when trying to deploy the containers). The recommended way > for handling isolated environments is to push containers to a local > registry on one of the hosts then set the local image as the image to use > during bootstrap (or afterwards with upgrade). See > https://docs.ceph.com/en/latest/cephadm/install/#deployment-in-an-isolated-environment > > - Adam King > > On Fri, Feb 4, 2022 at 6:10 AM Arun Vinod <arunvinod.tech@xxxxxxxxx> > wrote: > >> Hi All, >> >> We have a requirement for an offline ceph cluster deployment, where we >> will >> manually upload the cephadm container images and binaries and try to >> bootstrap the cluster using those. >> >> bootstrap command is: >> >> cephadm --image quay.io/ceph/ceph:v16.2.7 bootstrap >> --skip-monitoring-stack >> --mon-ip 10.175.41.11 --cluster-network 10.175.42.0/24 --ssh-user >> ceph_deploy --ssh-private-key /home/ceph_deploy/.ssh/id_rsa >> --ssh-public-key /home/ceph_deploy/.ssh/id_rsa.pub --config >> /home/ceph_deploy/ceph_bootstrap/ceph.conf --initial-dashboard-password >> GH60A78IXMJ7 --dashboard-password-noupdate --skip-pull >> >> expectation was with the --skip-pull flag, cephadm will not try to pull >> the >> image from the repository instead will use the image already available in >> the host. >> >> [root@hcictrl01 stack_orchestrator]# podman images >> REPOSITORY TAG IMAGE ID CREATED SIZE >> quay.io/ceph/ceph v16.2.7 231fd40524c4 2 days ago 1.39 GB >> >> However, cephadm is trying to pull the image from the quay.io repo even >> though the --skip-pull is provided and fails since it's an offline >> deployment. >> >> Some logs entries from cephadm while trying to bootstrap with the >> --skip-pull flag. >> >> 2022-02-04 16:26:03,116 7efeed3d5b80 DEBUG /usr/bin/ceph: Added host >> 'hcictrl01' with addr '10.175.41.11' >> 2022-02-04 16:26:03,512 7fa7047d1b80 DEBUG >> >> -------------------------------------------------------------------------------- >> cephadm ['--image', 'quay.io/ceph/ceph:v16.2.7', 'pull'] >> 2022-02-04 16:26:03,640 7fa7047d1b80 DEBUG /bin/podman: 3.3.1 >> 2022-02-04 16:26:03,712 7efeed3d5b80 INFO Deploying mon service with >> default placement... >> 2022-02-04 16:26:03,727 7fa7047d1b80 INFO Pulling container image >> quay.io/ceph/ceph:v16.2.7... >> 2022-02-04 16:26:03,877 7fa7047d1b80 DEBUG /bin/podman: Trying to pull >> quay.io/ceph/ceph:v16.2.7... >> 2022-02-04 16:26:03,879 7fa7047d1b80 DEBUG /bin/podman: Error: >> initializing >> source docker://quay.io/ceph/ceph:v16.2.7: pinging container registry >> quay.io: Get "https://quay.io/v2/": dial tcp: lookup quay.io on [::1]:53: >> read udp [::1]:50415->[::1]:53: read: connection refused >> 2022-02-04 16:26:03,884 7fa7047d1b80 INFO Non-zero exit code 125 from >> /bin/podman pull quay.io/ceph/ceph:v16.2.7 >> 2022-02-04 16:26:03,884 7fa7047d1b80 INFO /bin/podman: stderr Trying to >> pull quay.io/ceph/ceph:v16.2.7... >> 2022-02-04 16:26:03,884 7fa7047d1b80 INFO /bin/podman: stderr Error: >> initializing source docker://quay.io/ceph/ceph:v16.2.7: pinging container >> reg >> istry quay.io: Get "https://quay.io/v2/": dial tcp: lookup quay.io on >> [::1]:53: read udp [::1]:50415->[::1]:53: read: connection refused >> 2022-02-04 16:26:03,885 7fa7047d1b80 ERROR ERROR: Failed command: >> /bin/podman pull quay.io/ceph/ceph:v16.2.7 >> 2022-02-04 16:26:05,070 7efeed3d5b80 DEBUG /usr/bin/ceph: Scheduled mon >> update... >> 2022-02-04 16:26:05,435 7f036d3e5b80 DEBUG >> >> -------------------------------------------------------------------------------- >> cephadm ['--image', 'quay.io/ceph/ceph:v16.2.7', 'pull'] >> >> Is it expected behaviour from cephadm to still lookup the quay.io repo, >> even --skip-pull is provided and if so, how can we do cluster deployment >> where internet access is not available. >> >> Thanks in advance. >> >> Thanks and Regards, >> Arun Vinod >> _______________________________________________ >> ceph-users mailing list -- ceph-users@xxxxxxx >> To unsubscribe send an email to ceph-users-leave@xxxxxxx >> >> _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |