Hello all, I'd really appreciate some input from the more knowledgeable here. Is there a way I can access OSD objects if I have a BlueFS replay error? This error prevents me starting the OSD and also throws an error if I try using the bluestore or objectstore tools. - I can however run a ceph-bluestore-tool show-label without issue. I'm hoping there is another way or possibly a way to purge this log that I can still access the objects on this OSD. If deleting this reply log will help (even with some data loss) I'm happy to try it. This has caused a PG to go inactive and I'm considering deleting the PG and force re-creating it. - Saw this mentioned as a last resort option. Below is a snip of where things go wrong. - I don't know if there is even a chance or is this an unrecoverable state? 2023-01-25T10:05:26.543+0000 7fa773a14240 20 bluefs _replay 0x0: op_dir_link db/031664.sst to 29549 2023-01-25T10:05:26.543+0000 7fa773a14240 20 bluefs _replay 0x0: op_dir_link db/031665.sst to 29550 2023-01-25T10:05:26.543+0000 7fa773a14240 20 bluefs _replay 0x0: op_dir_link db/031666.sst to 29551 2023-01-25T10:05:26.543+0000 7fa773a14240 20 bluefs _replay 0x0: op_dir_link db/CURRENT to 29543 2023-01-25T10:05:26.543+0000 7fa773a14240 20 bluefs _replay 0x0: op_dir_link db/IDENTITY to 5 2023-01-25T10:05:26.543+0000 7fa773a14240 20 bluefs _replay 0x0: op_dir_link db/LOCK to 2 2023-01-25T10:05:26.543+0000 7fa773a14240 20 bluefs _replay 0x0: op_dir_link db/MANIFEST-031657 to 29542 2023-01-25T10:05:26.543+0000 7fa773a14240 20 bluefs _replay 0x0: op_dir_link db/OPTIONS-031645 to 29529 2023-01-25T10:05:26.543+0000 7fa773a14240 20 bluefs _replay 0x0: op_dir_link db/OPTIONS-031660 to 29545 2023-01-25T10:05:26.543+0000 7fa773a14240 20 bluefs _replay 0x0: op_dir_create db.slow 2023-01-25T10:05:26.543+0000 7fa773a14240 20 bluefs _replay 0x0: op_jump seq 5204712 offset 0x20000 2023-01-25T10:05:26.543+0000 7fa773a14240 10 bluefs _read h 0x55d2f1cfdb80 0x10000~10000 from file(ino 1 size 0x0 mtime 2022-10-07T17:55:34.189440+0000 allocated 420000 alloc_commit 420000 extents [1:0x1770170000~20000,1:0x53d1e900000~400000]) 2023-01-25T10:05:26.543+0000 7fa773a14240 20 bluefs _read left 0x10000 len 0x10000 2023-01-25T10:05:26.543+0000 7fa773a14240 20 bluefs _read got 65536 2023-01-25T10:05:26.543+0000 7fa773a14240 10 bluefs _read h 0x55d2f1cfdb80 0x20000~1000 from file(ino 1 size 0x20000 mtime 2022-10-07T17:55:34.189440+0000 allocated 420000 alloc_commit 420000 extents [1:0x1770170000~20000,1:0x53d1e900000~400000]) 2023-01-25T10:05:26.543+0000 7fa773a14240 20 bluefs _read fetching 0x0~100000 of 1:0x53d1e900000~400000 2023-01-25T10:05:26.547+0000 7fa773a14240 20 bluefs _read left 0x100000 len 0x1000 2023-01-25T10:05:26.547+0000 7fa773a14240 20 bluefs _read got 4096 2023-01-25T10:05:26.547+0000 7fa773a14240 10 bluefs _replay 0x20000: txn(seq 5204713 len 0x55 crc 0x81f48b1c) 2023-01-25T10:05:26.547+0000 7fa773a14240 20 bluefs _replay 0x20000: op_file_update file(ino 29551 size 0x0 mtime 2022-10-07T17:55:34.151007+0000 allocated 0 alloc_commit 0 extents []) 2023-01-25T10:05:26.547+0000 7fa773a14240 20 bluefs _replay 0x20000: op_dir_link db/031666.sst to 29551 2023-01-25T10:05:26.555+0000 7fa773a14240 -1 /build/ceph-17.2.5/src/os/bluestore/BlueFS.cc: In function 'int BlueFS::_replay(bool, bool)' thread 7fa773a14240 time 2023-01-25T10:05:26.551808+0000 /build/ceph-17.2.5/src/os/bluestore/BlueFS.cc: 1419: FAILED ceph_assert(r == q->second->file_map.end()) Kind regards Geoff _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |