On 22/08/2021 00.42, Torkil Svensgaard wrote:
Hi
Any suggestions as to the cause of this error? The device list seems
fine, a mix of already active OSDs and 4 empty, available drives.
There were 2 orphaned LVs on the db device. After I removed those the 4
available devices came up as OSDs. There were plenty of space on the db
device though, so unsure how the orphans came to block OSD creation.
Mvh.
Torkil
RuntimeError: Failed command: /usr/bin/docker run --rm --ipc=host
--stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume
--privileged --group-add=disk --init -e
CONTAINER_IMAGE=docker.io/ceph/ceph@sha256:829ebf54704f2d827de00913b171e5da741aad9b53c1f35ad59251524790eceb
-e NODE_NAME=dcn-ceph-03 -e CEPH_USE_RANDOM_NONCE=1 -e
CEPH_VOLUME_OSDSPEC_AFFINITY=slow -v
/var/run/ceph/3b7736c6-00e4-11ec-a3c5-3cecef467984:/var/run/ceph:z -v
/var/log/ceph/3b7736c6-00e4-11ec-a3c5-3cecef467984:/var/log/ceph:z -v
/var/lib/ceph/3b7736c6-00e4-11ec-a3c5-3cecef467984/crash:/var/lib/ceph/crash:z
-v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v
/run/lock/lvm:/run/lock/lvm -v
/var/lib/ceph/3b7736c6-00e4-11ec-a3c5-3cecef467984/selinux:/sys/fs/selinux:ro
-v /tmp/ceph-tmpee_fkmi9:/etc/ceph/ceph.conf:z -v
/tmp/ceph-tmpeapr11q1:/var/lib/ceph/bootstrap-osd/ceph.keyring:z
docker.io/ceph/ceph@sha256:829ebf54704f2d827de00913b171e5da741aad9b53c1f35ad59251524790eceb
lvm batch --no-auto /dev/sdaa /dev/sdab /dev/sdac /dev/sdh /dev/sdk
/dev/sdo /dev/sdv /dev/sdw /dev/sdx /dev/sdy /dev/sdz --db-devices
/dev/nvme0n1 --block-db-size 67000000000 --yes --no-systemd
Traceback (most recent call last):
File "/usr/share/ceph/mgr/cephadm/serve.py", line 582, in
_apply_all_services
if self._apply_service(spec):
File "/usr/share/ceph/mgr/cephadm/serve.py", line 639, in _apply_service
self.mgr.osd_service.create_from_spec(cast(DriveGroupSpec, spec))
File "/usr/share/ceph/mgr/cephadm/services/osd.py", line 73, in
create_from_spec
ret = create_from_spec_one(self.prepare_drivegroup(drive_group))
File "/usr/share/ceph/mgr/cephadm/utils.py", line 79, in
forall_hosts_wrapper
return CephadmOrchestrator.instance._worker_pool.map(do_work, vals)
File "/lib64/python3.6/multiprocessing/pool.py", line 266, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/lib64/python3.6/multiprocessing/pool.py", line 644, in get
raise self._value
File "/lib64/python3.6/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/lib64/python3.6/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "/usr/share/ceph/mgr/cephadm/utils.py", line 73, in do_work
return f(*arg)
File "/usr/share/ceph/mgr/cephadm/services/osd.py", line 65, in
create_from_spec_one
replace_osd_ids=osd_id_claims_for_host, env_vars=env_vars
File "/usr/share/ceph/mgr/cephadm/services/osd.py", line 91, in
create_single_host
code, '\n'.join(err)))
RuntimeError: cephadm exited with an error code: 1, stderr:Non-zero exit
code 1 from /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM
--net=host --entrypoint /usr/sbin/ceph-volume --privileged
--group-add=disk --init -e
CONTAINER_IMAGE=docker.io/ceph/ceph@sha256:829ebf54704f2d827de00913b171e5da741aad9b53c1f35ad59251524790eceb
-e NODE_NAME=dcn-ceph-03 -e CEPH_USE_RANDOM_NONCE=1 -e
CEPH_VOLUME_OSDSPEC_AFFINITY=slow -v
/var/run/ceph/3b7736c6-00e4-11ec-a3c5-3cecef467984:/var/run/ceph:z -v
/var/log/ceph/3b7736c6-00e4-11ec-a3c5-3cecef467984:/var/log/ceph:z -v
/var/lib/ceph/3b7736c6-00e4-11ec-a3c5-3cecef467984/crash:/var/lib/ceph/crash:z
-v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v
/run/lock/lvm:/run/lock/lvm -v
/var/lib/ceph/3b7736c6-00e4-11ec-a3c5-3cecef467984/selinux:/sys/fs/selinux:ro
-v /tmp/ceph-tmpee_fkmi9:/etc/ceph/ceph.conf:z -v
/tmp/ceph-tmpeapr11q1:/var/lib/ceph/bootstrap-osd/ceph.keyring:z
docker.io/ceph/ceph@sha256:829ebf54704f2d827de00913b171e5da741aad9b53c1f35ad59251524790eceb
lvm batch --no-auto /dev/sdaa /dev/sdab /dev/sdac /dev/sdh /dev/sdk
/dev/sdo /dev/sdv /dev/sdw /dev/sdx /dev/sdy /dev/sdz --db-devices
/dev/nvme0n1 --block-db-size 67000000000 --yes --no-systemd
/usr/bin/docker: stderr --> passed data devices: 11 physical, 0 LVM
/usr/bin/docker: stderr --> relative data size: 1.0
/usr/bin/docker: stderr --> passed block_db devices: 1 physical, 0 LVM
/usr/bin/docker: stderr --> 2 fast allocations != 4 num_osds
Traceback (most recent call last):
File
"/var/lib/ceph/3b7736c6-00e4-11ec-a3c5-3cecef467984/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931",
line 8230, in <module>
main()
File
"/var/lib/ceph/3b7736c6-00e4-11ec-a3c5-3cecef467984/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931",
line 8218, in main
r = ctx.func(ctx)
File
"/var/lib/ceph/3b7736c6-00e4-11ec-a3c5-3cecef467984/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931",
line 1653, in _infer_fsid
return func(ctx)
File
"/var/lib/ceph/3b7736c6-00e4-11ec-a3c5-3cecef467984/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931",
line 1737, in _infer_image
return func(ctx)
File
"/var/lib/ceph/3b7736c6-00e4-11ec-a3c5-3cecef467984/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931",
line 4599, in command_ceph_volume
out, err, code = call_throws(ctx, c.run_cmd())
File
"/var/lib/ceph/3b7736c6-00e4-11ec-a3c5-3cecef467984/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931",
line 1453, in call_throws
raise RuntimeError('Failed command: %s' % ' '.join(command))
RuntimeError: Failed command: /usr/bin/docker run --rm --ipc=host
--stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume
--privileged --group-add=disk --init -e
CONTAINER_IMAGE=docker.io/ceph/ceph@sha256:829ebf54704f2d827de00913b171e5da741aad9b53c1f35ad59251524790eceb
-e NODE_NAME=dcn-ceph-03 -e CEPH_USE_RANDOM_NONCE=1 -e
CEPH_VOLUME_OSDSPEC_AFFINITY=slow -v
/var/run/ceph/3b7736c6-00e4-11ec-a3c5-3cecef467984:/var/run/ceph:z -v
/var/log/ceph/3b7736c6-00e4-11ec-a3c5-3cecef467984:/var/log/ceph:z -v
/var/lib/ceph/3b7736c6-00e4-11ec-a3c5-3cecef467984/crash:/var/lib/ceph/crash:z
-v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v
/run/lock/lvm:/run/lock/lvm -v
/var/lib/ceph/3b7736c6-00e4-11ec-a3c5-3cecef467984/selinux:/sys/fs/selinux:ro
-v /tmp/ceph-tmpee_fkmi9:/etc/ceph/ceph.conf:z -v
/tmp/ceph-tmpeapr11q1:/var/lib/ceph/bootstrap-osd/ceph.keyring:z
docker.io/ceph/ceph@sha256:829ebf54704f2d827de00913b171e5da741aad9b53c1f35ad59251524790eceb
lvm batch --no-auto /dev/sdaa /dev/sdab /dev/sdac /dev/sdh /dev/sdk
/dev/sdo /dev/sdv /dev/sdw /dev/sdx /dev/sdy /dev/sdz --db-devices
/dev/nvme0n1 --block-db-size 67000000000 --yes --no-systemd
This is the OSD spec which works just fine for for the two other similar
hosts in the cluster:
service_type: osd
service_id: slow
placement:
host_pattern: '*'
data_devices:
rotational: 1
# size: '10000G:'
db_devices:
rotational: 0
size: '1000G:'
db_slots: 22
block_db_size: 67000000000
Thanks,
Torkil
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
