Discussion on issues encountered while creating osds

[hera sami <herasami28mnnit@xxxxxxxxx>]

Hi team, glad to connect with you all.

I am seeing an issue with ceph deployed using cephadm. I am using the "reef" version of ceph.

I had written an ansible script for bringing up ceph as described in the ceph official documentation. My script works fine most of the time but on some vms I am seeing an issue.

I am creating a cluster of 3 nodes. The bootstrapping went well and other 2 nodes got added to the cluster.

While my mon and mgr are added successfully, the osd on 2 nodes are not coming up.

While the device for osd creation is AVAILABLE as seen from below, when i run the ceph orch add osd command, the ceph orch command hangs indefinitely. 

I am trying to create osd on sftp-ntp-1 and ceph-non-sftp-1 with device /dev/sdb.

Althought the device is AVAILABLE, but when I run the ceph och add osd command, the ceph orch command hangs indefinitely. Neither the osd gets added nor any ceph orch or ceph osd commands accessible.

Sometimes ceph orch commands resumes when executed "sudo systemctl restart ceph.target", but no luck on osd creation.

Also in these nodes, I dont see the logs of Reconfig daemon osd in /var/log/ceph/cephadm.log unlike other daemons like mon, mgr 

root@sftp-ntp-1:/var/log# cat /var/log/ceph/cephadm.log | grep "Reconfig daemon"

2025-02-27 06:49:09,426 7f37b62ce1c0 INFO Reconfig daemon mgr.sftp-ntp-1.hkvpch ...

2025-02-27 06:49:09,807 7fbbcc9d21c0 INFO Reconfig daemon mon.sftp-ntp-1 ...

2025-02-27 06:49:10,203 7f9cd5a691c0 INFO Reconfig daemon crash.sftp-ntp-1 ...

2025-02-27 06:49:10,581 7fbe2e4e11c0 INFO Reconfig daemon ceph-exporter.sftp-ntp-1 ...

This is seen on only few vms and I am not able to understand what is the root cause behind the issue.

It would be much appreciated if I can get any sort of idea on this.

Thanks,

Hera

_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx