I've seen what appears to be the same post on Reddit, previously, and attempted to assist. My suspicion is a "stop" command was passed to ceph orch upgrade in an attempt to stop it, but with the --image flag preceding it, setting the image to stop. I asked the user to do an actual upgrade stop, then re-attempt specifying a different image, and the user indicated the "stop" image pull attempt continued. That part didn't seem right, to which I suggested a bug report. https://www.reddit.com/r/ceph/comments/11g3rze/anyone_having_pull_issues_with_ceph_images/ @OP - are you the same poster as the above, or do you just have the same problem? If there's multiple users with this, it would indicate something larger than just a misplaced option/flag/command. If it is you - could you link to the bug report? Just to make sure, you've issued: "ceph orch upgrade stop" Then performed another "ceph orch upgrade start" specifying a --ceph-version or --image? I'll also echo Adam's request for a "ceph config dump |grep image". It sounds like it's still set to "stop", but I'd have expected the above to initiate an upgrade to the correct image. If not, the bug report would be helpful to continue so it could be fixed. David On Mon, Mar 6, 2023, at 15:02, Adam King wrote: > Can I see the output of `ceph orch upgrade status` and `ceph config dump | > grep image`? The "Pulling container image stop" implies somehow (as Eugen > pointed out) that cephadm thinks the image to pull is named "stop" which > means it is likely set as either the image to upgrade to or as one of the > config options. > > On Sat, Mar 4, 2023 at 2:06 AM <aellahib@xxxxxxxxx> wrote: > >> I initially ran the upgrade fine but it failed @ around 40/100 on an osd, >> so after waiting for along time i thought I'd try restarting it and then >> restarting the upgrade. >> I am stuck with the below debug error, I have tested docker pull from >> other servers and they dont fail for the ceph images but on ceph it does. >> If i even try to redeploy or add or remove mon damons for example it comes >> up with the same error related to the images. >> >> The error that ceph is giving me is: >> 2023-03-02T07:22:45.063976-0700 mgr.mgr-node.idvkbw [DBG] _run_cephadm : >> args = [] >> 2023-03-02T07:22:45.070342-0700 mgr.mgr-node.idvkbw [DBG] args: --image >> stop --no-container-init pull >> 2023-03-02T07:22:45.081086-0700 mgr.mgr-node.idvkbw [DBG] Running command: >> which python3 >> 2023-03-02T07:22:45.180052-0700 mgr.mgr-node.idvkbw [DBG] Running command: >> /usr/bin/python3 >> /var/lib/ceph/5058e342-dac7-11ec-ada3-01065e90228d/cephadm.059bfc99f5cf36ed881f2494b104711faf4cbf5fc86a9594423cc105cafd9b4e >> --image stop --no-container-init pull >> 2023-03-02T07:22:46.500561-0700 mgr.mgr-node.idvkbw [DBG] code: 1 >> 2023-03-02T07:22:46.500787-0700 mgr.mgr-node.idvkbw [DBG] err: Pulling >> container image stop... >> Non-zero exit code 1 from /usr/bin/docker pull stop >> /usr/bin/docker: stdout Using default tag: latest >> /usr/bin/docker: stderr Error response from daemon: pull access denied for >> stop, repository does not exist or may require 'docker login': denied: >> requested access to the resource is denied >> ERROR: Failed command: /usr/bin/docker pull stop >> _______________________________________________ >> ceph-users mailing list -- ceph-users@xxxxxxx >> To unsubscribe send an email to ceph-users-leave@xxxxxxx >> >> > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |