Hi folks, I'm trying to install ceph on GCE VMs (debian/ubuntu) with PD-SSDs using ceph-ansible image. The installation from clean has been good, but when I purged ceph cluster and tried to re-install, I saw the error: ``` Running command: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 1 --monmap /var/lib/ceph/osd/ceph-1/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-1/ --osd-uuid 08d766d5-e843-4c65-9f4f-db7f0129b4e9 --setuser ceph --setgroup ceph stderr: 2022-10-21T22:07:58.447+0000 7f71afead080 -1 bluestore(/var/lib/ceph/osd/ceph-1/) _read_fsid unparsable uuid stderr: 2022-10-21T22:07:58.455+0000 7f71afead080 -1 bluefs _replay 0x0: stop: uuid 8b1ce55d-10c1-a33d-1817-8a8427657694 != super.uuid 3d8aa673-00bd-473c-a725-06ac31c6b945, block dump: stderr: 00000000 6a bc c7 44 83 87 8b 1c e5 5d 10 c1 a3 3d 18 17 |j..D.....]...=..| stderr: 00000010 8a 84 27 65 76 94 bd 12 3c 11 4a c4 32 6c eb a4 |..'ev...<.J.2l..| … stderr: 00000ff0 2b 57 4e a4 ad da be cb bf df 61 fc f7 ce 4a 14 |+WN.......a...J.| stderr: 00001000 stderr: 2022-10-21T22:07:58.987+0000 7f71afead080 -1 rocksdb: verify_sharding unable to list column families: NotFound: stderr: 2022-10-21T22:07:58.987+0000 7f71afead080 -1 bluestore(/var/lib/ceph/osd/ceph-1/) _open_db erroring opening db: stderr: 2022-10-21T22:07:59.515+0000 7f71afead080 -1 OSD::mkfs: ObjectStore::mkfs failed with error (5) Input/output error stderr: 2022-10-21T22:07:59.515+0000 7f71afead080 -1 [0;31m ** ERROR: error creating empty object store in /var/lib/ceph/osd/ceph-1/: (5) Input/output error[0m --> Was unable to complete a new OSD, will rollback changes ``` Can someone explain what uuid != super.uuid means? The issue seems not to happen when installing on a clean disk. Would it be related to the purging process not doing a good cleanup job? FWIW, I'm using https://github.com/ceph/ceph-ansible/blob/main/infrastructure-playbooks/purge-cluster.yml to purge the cluster. Thanks, Zhongzhou Cai _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |