| Hi, A few months back, we had one of our OSD node motherboards die. At the time, we simply waited for recovery and purged the OSDs that were on the dead node. We just replaced that node and added back the drives as new OSDs. At the ceph administration level, everything looks fine, no duplicate OSDs when I execute map commands or ask Ceph to list what OSDs are on the node. However, on the OSD node, in /var/log/ceph/ceph-volume, I see that every time the server boots, ceph-volume tries to search for OSD fsids that don’t exist. Here’s the error: [2019-10-29 13:12:02,864][ceph_volume][ERROR ] exception caught by decorator Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ceph_volume/decorators.py", line 59, in newfunc return f(*a, **kw) File "/usr/lib/python2.7/site-packages/ceph_volume/main.py", line 148, in main terminal.dispatch(self.mapper, subcommand_args) File "/usr/lib/python2.7/site-packages/ceph_volume/terminal.py", line 182, in dispatch instance.main() File "/usr/lib/python2.7/site-packages/ceph_volume/devices/lvm/main.py", line 40, in main terminal.dispatch(self.mapper, self.argv) File "/usr/lib/python2.7/site-packages/ceph_volume/terminal.py", line 182, in dispatch instance.main() File "/usr/lib/python2.7/site-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python2.7/site-packages/ceph_volume/devices/lvm/trigger.py", line 70, in main Activate(['--auto-detect-objectstore', osd_id, osd_uuid]).main() File "/usr/lib/python2.7/site-packages/ceph_volume/devices/lvm/activate.py", line 339, in main self.activate(args) File "/usr/lib/python2.7/site-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python2.7/site-packages/ceph_volume/devices/lvm/activate.py", line 249, in activate raise RuntimeError('could not find osd.%s with fsid %s' % (osd_id, osd_fsid)) RuntimeError: could not find osd.213 with fsid 22800a80-2445-41a3-8643-69b4b84d598a Of course this fsid ID isn’t listed anywhere in Ceph. Where does ceph-volume get this fsid from? Even when looking at the code, it’s not particularly obvious. This is ceph mimic running on CentOS 7 and bluestore. Jean-Philippe Méthot Openstack system administrator Administrateur système Openstack PlanetHoster inc. |
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
