Hi Gents, thanks for your replies. I have stopped the removal command and reinitiated it again. To my surprise it has removed the bucket pretty quickly this time around. Not too sure if it has removed all the objects or orphans. I will need to further investigate this matter. Andrei ----- Original Message ----- > From: "Matt Benjamin" <mbenjami@xxxxxxxxxx> > To: "EDH" <mriosfer@xxxxxxxxxxxxxxxx> > Cc: "Andrei Mikhailovsky" <andrei@xxxxxxxxxx>, "ceph-users" <ceph-users@xxxxxxx>, "Ivancich, Eric" <ivancich@xxxxxxxxxx> > Sent: Thursday, 6 August, 2020 21:48:53 > Subject: Re: RGW unable to delete a bucket > Hi Folks, > > I don't know of a downstream issue that looks like this, and we've > upstreamed every fix for bucket listing and cleanup we have. We are > pursuing a space leak believed to arise in "radosgw-admin bucket rm > --purge-objects" but not a non-terminating listing. > > The only upstream release not planned to get a backport of orphans > list tools is Luminous. I thought backport to Octopus was already > done by the backport team? > > regards, > > Matt > > On Thu, Aug 6, 2020 at 2:40 PM EDH - Manuel Rios > <mriosfer@xxxxxxxxxxxxxxxx> wrote: >> >> You'r not the only one affected by this issue >> >> As far as i know several huge companies hitted this bug too, but private patches >> or tools are not public released. >> >> This is caused for the a resharding process during upload in previous versions. >> >> Workarround for us.: >> >> - Delete objects of the bucket at rados level. >> - Delete the index file of the bucket. >> >> Pray to god to not happen again. >> >> Still pending backporting to Nautilus of the new experimental tool to find >> orphans in RGW >> >> Maybe @Matt Benjamin can give us and ETA for get ready that tool backported... >> >> Regards >> >> >> >> -----Mensaje original----- >> De: Andrei Mikhailovsky <andrei@xxxxxxxxxx> >> Enviado el: jueves, 6 de agosto de 2020 13:55 >> Para: ceph-users <ceph-users@xxxxxxx> >> Asunto: Re: RGW unable to delete a bucket >> >> BUMP... >> >> >> ----- Original Message ----- >> > From: "Andrei Mikhailovsky" <andrei@xxxxxxxxxx> >> > To: "ceph-users" <ceph-users@xxxxxxx> >> > Sent: Tuesday, 4 August, 2020 17:16:28 >> > Subject: RGW unable to delete a bucket >> >> > Hi >> > >> > I am trying to delete a bucket using the following command: >> > >> > # radosgw-admin bucket rm --bucket=<bucket-name> --purge-objects >> > >> > However, in console I get the following messages. About 100+ of those >> > messages per second. >> > >> > 2020-08-04T17:11:06.411+0100 7fe64cacf080 1 >> > RGWRados::Bucket::List::list_objects_ordered INFO ordered bucket >> > listing requires read #1 >> > >> > >> > The command has been running for about 35 days days and it still >> > hasn't finished. The size of the bucket is under 1TB for sure. Probably around >> > 500GB. >> > >> > I have recently removed about a dozen of old buckets without any >> > issues. It's this particular bucket that is being very stubborn. >> > >> > Anything I can do to remove it, including it's objects and any orphans >> > it might have? >> > >> > >> > Thanks >> > >> > Andrei >> > _______________________________________________ >> > ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an >> > email to ceph-users-leave@xxxxxxx >> _______________________________________________ >> ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to >> ceph-users-leave@xxxxxxx >> > > > -- > > Matt Benjamin > Red Hat, Inc. > 315 West Huron Street, Suite 140A > Ann Arbor, Michigan 48103 > > http://www.redhat.com/en/technologies/storage > > tel. 734-821-5101 > fax. 734-769-8938 > cel. 734-216-5309 _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |