You’ve provided convincing evidence that the bucket index is not correctly reflecting the data objects. So the next step would be to remove the bucket index entries for these 39 objects. It looks like you’ve already mapped which entries go to which bucket index shards (or you could redo your commands to get that info). So you could use `rados rmomapkey…` to get rid of those. Sometimes bucket index entries have non-printable characters in their keys, which makes it challenging to provide the key on the command-line. In such cases you can instead put the key in a file and then use the "--omap-key-file” command-line option to refer to that file. I realize this is a pain and I’m sorry that you have to go through this. But once you remove those bucket index entries you should be all set. Eric (he/him) > On May 9, 2022, at 4:20 PM, Christopher Durham <caduceus42@xxxxxxx> wrote: > > > I am using pacific 16.2.7 on rocket 8.5 Linux 8.5. > I have a once heavily used radosgw bucket that is now empty. Let's call it 'oldbucket' awscli now shows that there are no objects in the bucket. > > However, radosgw-admin bucket stats --bucket oldbucket shows num_objects in rgw.main as 39 objects, with about 200 gigs used in the size field. > > radosgw-admin bucket check --bucket oldbucket indeed shows 39 objects in a list. > Each of these objects are of the form: > _multipart_originalobjectname.someoriginalid.NN > radosgw-admin bucket radoslist --bucket oldbucket shows only 9 objects, but thoseobjects are alll included as part of the bucket check command above. > This particular bucket did have alot of multipart uploads. All multipart uploads have beenaborted with awscli. And of course awscli shows no objects in the bucket > > radosgw-admin bucket list --bucket oldbucket shows all 39 objects, which is weird thatI see object names which once were multipart object parts. > > None of the 39 objects exist in the pool (rados -p $pool get $obj /tmp/$obj all return 1) > If I list all the index objects for this bucket in the index pool and then do a listomapkeys for each of the index objects,I see only the 39 omapkeys. > So my question is, what do I need to do to fix this bucket (without delting and recreating it?) Would just doing a rmomapkeyon each of the omap keys in the listomapkeys output solve my problem, and reclaim the 200 Gigs? > Do I have to rebuild the index somehow (--fix in bucket check above did nothing). > > Thanks for any thoughts/insight. > -Chris > > > > > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |