I've met this issue when try to upgrade octopus 15.2.17 to 16.2.13 last night. Upgrade process failed at mgr module phase after the new MGR version become to active state. I tried to enable debug `ceph config set mgr mgr/cephadm/log_to_cluster_level debug ` and I saw the message like @xadhoom76 about config_key "registry_credentials" I guessed the root cause because of this line `json.loads(str(self.mgr.get_store('registry_credentials'))))` and the key_store had a wrong value. Then I got the empty value when run this command "ceph config-key dump | grep 'registry_credentials'" and the same for "ceph config-key get mgr/cephadm/registry_credentials" By check `cephadm` source i see the value should be a json format like that ceph config-key set mgr/cephadm/registry_credentials '{"url": "registry.local:5000", "username": "user-deployer", "password": "xxxyyyyzzzzzz"}' After set this key and `ceph mgr fail` to reload , my cluster issue was gone . _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx