There is a problem upgrading ceph-iscsi from 16.25 to 16.2.6
2021-09-21T12:43:58.767556-0400 mgr.nj3231.wagzhn [ERR] cephadm exited
with an error code: 1, stderr:Redeploy daemon iscsi.iscsi.nj3231.mqeari ...
Creating ceph-iscsi config...
Write file:
/var/lib/ceph/c6c8bc66-1716-11ec-b029-1c34da4b9fb6/iscsi.iscsi.nj3231.mqeari/iscsi-gateway.cfg
Failed to trim old cgroups
/sys/fs/cgroup/system.slice/system-ceph\x2dc6c8bc66\x2d1716\x2d11ec\x2db029\x2d1c34da4b9fb6.slice/ceph-c6c8bc66-1716-11ec-b029-1c34da4b9fb6@iscsi.iscsi.nj3231.mqeari.service
Non-zero exit code 1 from systemctl start
ceph-c6c8bc66-1716-11ec-b029-1c34da4b9fb6@iscsi.iscsi.nj3231.mqeari
systemctl: stderr Job for
ceph-c6c8bc66-1716-11ec-b029-1c34da4b9fb6@iscsi.iscsi.nj3231.mqeari.service
failed because the control process exited with error code.
systemctl: stderr See "systemctl status
ceph-c6c8bc66-1716-11ec-b029-1c34da4b9fb6@iscsi.iscsi.nj3231.mqeari.service"
and "journalctl -xe" for details.
Traceback (most recent call last):
File
"/var/lib/ceph/c6c8bc66-1716-11ec-b029-1c34da4b9fb6/cephadm.f46dc95b01feeedb28941a48e2f1d0abb51139ca828de11150ea7122a8e3549c",
line 8479, in <module>
main()
File
"/var/lib/ceph/c6c8bc66-1716-11ec-b029-1c34da4b9fb6/cephadm.f46dc95b01feeedb28941a48e2f1d0abb51139ca828de11150ea7122a8e3549c",
line 8467, in main
r = ctx.func(ctx)
File
"/var/lib/ceph/c6c8bc66-1716-11ec-b029-1c34da4b9fb6/cephadm.f46dc95b01feeedb28941a48e2f1d0abb51139ca828de11150ea7122a8e3549c",
line 1782, in _default_image
return func(ctx)
File
"/var/lib/ceph/c6c8bc66-1716-11ec-b029-1c34da4b9fb6/cephadm.f46dc95b01feeedb28941a48e2f1d0abb51139ca828de11150ea7122a8e3549c",
line 4523, in command_deploy
deploy_daemon(ctx, ctx.fsid, daemon_type, daemon_id, c, uid, gid,
File
"/var/lib/ceph/c6c8bc66-1716-11ec-b029-1c34da4b9fb6/cephadm.f46dc95b01feeedb28941a48e2f1d0abb51139ca828de11150ea7122a8e3549c",
line 2669, in deploy_daemon
deploy_daemon_units(ctx, fsid, uid, gid, daemon_type, daemon_id,
File
"/var/lib/ceph/c6c8bc66-1716-11ec-b029-1c34da4b9fb6/cephadm.f46dc95b01feeedb28941a48e2f1d0abb51139ca828de11150ea7122a8e3549c",
line 2899, in deploy_daemon_units
call_throws(ctx, ['systemctl', 'start', unit_name])
File
"/var/lib/ceph/c6c8bc66-1716-11ec-b029-1c34da4b9fb6/cephadm.f46dc95b01feeedb28941a48e2f1d0abb51139ca828de11150ea7122a8e3549c",
line 1462, in call_throws
raise RuntimeError('Failed command: %s' % ' '.join(command))
RuntimeError: Failed command: systemctl start
ceph-c6c8bc66-1716-11ec-b029-1c34da4b9fb6@iscsi.iscsi.nj3231.mqeari
Traceback (most recent call last):
File "/usr/share/ceph/mgr/cephadm/serve.py", line 1366, in
_remote_connection
yield (conn, connr)
File "/usr/share/ceph/mgr/cephadm/serve.py", line 1263, in _run_cephadm
code, '\n'.join(err)))
orchestrator._interface.OrchestratorError: cephadm exited with an error
code: 1, stderr:Redeploy daemon iscsi.iscsi.nj3231.mqeari ...
Creating ceph-iscsi config...
Write file:
/var/lib/ceph/c6c8bc66-1716-11ec-b029-1c34da4b9fb6/iscsi.iscsi.nj3231.mqeari/iscsi-gateway.cfg
Failed to trim old cgroups
/sys/fs/cgroup/system.slice/system-ceph\x2dc6c8bc66\x2d1716\x2d11ec\x2db029\x2d1c34da4b9fb6.slice/ceph-c6c8bc66-1716-11ec-b029-1c34da4b9fb6@iscsi.iscsi.nj3231.mqeari.service
Non-zero exit code 1 from systemctl start
ceph-c6c8bc66-1716-11ec-b029-1c34da4b9fb6@iscsi.iscsi.nj3231.mqeari
systemctl: stderr Job for
ceph-c6c8bc66-1716-11ec-b029-1c34da4b9fb6@iscsi.iscsi.nj3231.mqeari.service
failed because the control process exited with error code.
systemctl: stderr See "systemctl status
ceph-c6c8bc66-1716-11ec-b029-1c34da4b9fb6@iscsi.iscsi.nj3231.mqeari.service"
and "journalctl -xe" for details.
Traceback (most recent call last):
File
"/var/lib/ceph/c6c8bc66-1716-11ec-b029-1c34da4b9fb6/cephadm.f46dc95b01feeedb28941a48e2f1d0abb51139ca828de11150ea7122a8e3549c",
line 8479, in <module>
main()
File
"/var/lib/ceph/c6c8bc66-1716-11ec-b029-1c34da4b9fb6/cephadm.f46dc95b01feeedb28941a48e2f1d0abb51139ca828de11150ea7122a8e3549c",
line 8467, in main
r = ctx.func(ctx)
File
"/var/lib/ceph/c6c8bc66-1716-11ec-b029-1c34da4b9fb6/cephadm.f46dc95b01feeedb28941a48e2f1d0abb51139ca828de11150ea7122a8e3549c",
line 1782, in _default_image
return func(ctx)
File
"/var/lib/ceph/c6c8bc66-1716-11ec-b029-1c34da4b9fb6/cephadm.f46dc95b01feeedb28941a48e2f1d0abb51139ca828de11150ea7122a8e3549c",
line 4523, in command_deploy
deploy_daemon(ctx, ctx.fsid, daemon_type, daemon_id, c, uid, gid,
File
"/var/lib/ceph/c6c8bc66-1716-11ec-b029-1c34da4b9fb6/cephadm.f46dc95b01feeedb28941a48e2f1d0abb51139ca828de11150ea7122a8e3549c",
line 2669, in deploy_daemon
deploy_daemon_units(ctx, fsid, uid, gid, daemon_type, daemon_id,
File
"/var/lib/ceph/c6c8bc66-1716-11ec-b029-1c34da4b9fb6/cephadm.f46dc95b01feeedb28941a48e2f1d0abb51139ca828de11150ea7122a8e3549c",
line 2899, in deploy_daemon_units
call_throws(ctx, ['systemctl', 'start', unit_name])
File
"/var/lib/ceph/c6c8bc66-1716-11ec-b029-1c34da4b9fb6/cephadm.f46dc95b01feeedb28941a48e2f1d0abb51139ca828de11150ea7122a8e3549c",
line 1462, in call_throws
raise RuntimeError('Failed command: %s' % ' '.join(command))
RuntimeError: Failed command: systemctl start
ceph-c6c8bc66-1716-11ec-b029-1c34da4b9fb6@iscsi.iscsi.nj3231.mqeari
On 18.9.2021 г. 10:02 ч., Sean wrote:
They really need to update the upgrade doc, as they’ve quit using Docker
Hub for images (https://hub.docker.com/r/ceph/ceph). I can’t say I blame
them, considering the silliness that Docker has been up to.
Anyway, you need to specify the quay.io registry:
ceph orch upgrade start --image quay.io/ceph/ceph:v16.2.6
~ Sean
On Sep 18, 2021 at 6:02:06 AM, Cem Zafer <cemzafer@xxxxxxxxx> wrote:
Here is the detail error.
Thanks.
root@ceph100:~# ceph health detail
HEALTH_WARN Upgrade: failed to pull target image
[WRN] UPGRADE_FAILED_PULL: Upgrade: failed to pull target image
host ceph100 `cephadm pull` failed: cephadm exited with an error code:
1, stderr:Pulling container image docker.io/ceph/ceph:v16.2.6...
Non-zero exit code 1 from /usr/bin/docker pull docker.io/ceph/ceph:v16.2.6
/usr/bin/docker: stderr Error response from daemon: manifest for
ceph/ceph:v16.2.6 not found: manifest unknown: manifest unknown
Traceback (most recent call last):
File
"/var/lib/ceph/96825634-ff47-11eb-9c13-d7c69fddf094/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931",
line 8230, in <module>
main()
File
"/var/lib/ceph/96825634-ff47-11eb-9c13-d7c69fddf094/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931",
line 8218, in main
r = ctx.func(ctx)
File
"/var/lib/ceph/96825634-ff47-11eb-9c13-d7c69fddf094/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931",
line 1737, in _infer_image
return func(ctx)
File
"/var/lib/ceph/96825634-ff47-11eb-9c13-d7c69fddf094/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931",
line 3286, in command_pull
_pull_image(ctx, ctx.image)
File
"/var/lib/ceph/96825634-ff47-11eb-9c13-d7c69fddf094/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931",
line 3311, in _pull_image
raise RuntimeError('Failed command: %s' % cmd_str)
RuntimeError: Failed command: /usr/bin/docker pull
docker.io/ceph/ceph:v16.2.6
On Sat, Sep 18, 2021 at 1:56 PM Cem Zafer <cemzafer@xxxxxxxxx> wrote:
Hi,
As usual upgrade procedurefrom 16.2.5 -> 16.2.6, I get that error. Anyone
have any suggestions?
Thanks.
root@ceph100:~# ceph orch upgrade status
{
"target_image": "docker.io/ceph/ceph:v16.2.6",
"in_progress": true,
"services_complete": [],
"progress": "",
"message": "Error: UPGRADE_FAILED_PULL: Upgrade: failed to pull target
image"
}
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
