Re: safe to remove leftover bucket index objects

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



I'm glad you asked this, because it was on my to-do list. I know that based on our not existing in the bucket marker does not mean it's safe to delete.  I have an index pool with 22k objects in it. 70 objects match existing bucket markers. I was having a problem on the cluster and started deleting the objects in the index pool and after going through 200 objects I stopped it and tested and list access to 3 pools. Luckily for me they were all buckets I've been working on deleting, so no need for recovery.

I then compared bucket IDs to the objects in that pool, but still only found a couple hundred more matching objects. I have no idea what the other 22k objects are in the index bucket that don't match bucket markers or bucket IDs. I did confirm there was no resharding happening both in the research list and all bucket reshard statuses.

Does anyone know how to parse the names of these objects and how to tell what can be deleted?  This is if particular interest as I have another costed with 1M injects in the index pool.

On Thu, Aug 30, 2018, 7:29 AM Dan van der Ster <dan@xxxxxxxxxxxxxx> wrote:
Replying to self...

On Wed, Aug 1, 2018 at 11:56 AM Dan van der Ster <dan@xxxxxxxxxxxxxx> wrote:
>
> Dear rgw friends,
>
> Somehow we have more than 20 million objects in our
> default.rgw.buckets.index pool.
> They are probably leftover from this issue we had last year:
> http://lists.ceph.com/pipermail/ceph-users-ceph.com/2017-June/018565.html
> and we want to clean the leftover / unused index objects
>
> To do this, I would rados ls the pool, get a list of all existing
> buckets and their current marker, then delete any objects with an
> unused marker.
> Does that sound correct?

More precisely, for example, there is an object
.dir.61c59385-085d-4caa-9070-63a3868dccb6.2978181.59.8 in the index
pool.
I run `radosgw-admin bucket stats` to get the marker for all current
existing buckets.
The marker 61c59385-085d-4caa-9070-63a3868dccb6.2978181.59 is not
mentioned in the bucket stats output.
Is it safe to rados rm .dir.61c59385-085d-4caa-9070-63a3868dccb6.2978181.59.8 ??

Thanks in advance!

-- dan







> Can someone suggest a better way?
>
> Cheers, Dan
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux