Hi everybody, are there other ways for rados objects to get removed, other than "rados -p POOL rm OBJECT"? We have a customer who got objects in the bucket index, but can't download it. After checking it seems like the rados object is gone. Ceph cluster is running ceph octopus 15.2.16 "radosgw-admin bi list --bucket BUCKET" shows the object available. "radosgw-admin bucket radoslist --bucket BUCKET" shows the object and a corresponding multipart file. "rados -p POOL ls" only shows the object, but not the multipart file. Exporting the rados object hands me an empty file. I find it very strange that a 250KB file get a multipart object, but what do I know how the customer uploaded the file and how they work with the RGW api. What grinds my gears is that we lost customer data, and I need to know what ways are there that leads to said problem. I know there is no recovery, but I am not satisfied with "well, it just happened. No idea why". As I am the only one who is working on the the ceph cluster I would remove "removed via rados command" from the list of possibilities, as the last orphan objects cleanup was performed a month before the files last MTIME. Is there ANY way this could happen in some correlation with the GC, restarting/adding/removing OSDs, sharding bucket indexes, OSD crashes and other? Anything that isn't "rados -p POOL rm OBJECT"? Cheers Boris _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |