Hi Boris, The objects do exist and I can see it with ls. I can also verify that the total amount of objects in the pool is over 2m more than the amount of files. The total used space of all the buckets is about 10TB less than the total space used up by the .rgw.buckets pool. My colleague has suggested that there are unprintable characters in the object names and thus they can't be removed with cli tools. Could this be the case and if so, how do I remove them? Cheers Andrei ----- Original Message ----- > From: "Boris Behrens" <bb@xxxxxxxxx> > To: "ceph-users" <ceph-users@xxxxxxx> > Sent: Tuesday, 3 January, 2023 12:53:29 > Subject: Re: rgw - unable to remove some orphans > Hi Andrei, > happy new year to you too. > > The file might be already removed. > You can check if the radosobject is there with `rados -p ls ...` > You can also check if the file is is still in the bucket with > `radosgw-admin bucket radoslist --bucket BUCKET` > > Cheers > Boris > > Am Di., 3. Jan. 2023 um 13:47 Uhr schrieb Andrei Mikhailovsky > <andrei@xxxxxxxxxx>: >> >> Happy New Year everyone! >> >> I have a bit of an issue with removing some of the orphan objects that were >> generated with the rgw-orphan-list tool. Over the years rgw generated over 14 >> million orphans with an overall waste of over 100TB in size, considering the >> overall data stored in rgw was well under 10TB at max. Anyways, I have managed >> to remove around 12m objects over the holiday season, but there are just over >> 2m orphans which were not removed. Here is an example of one of the objects >> taken from the orphans list file: >> >> $ rados -p .rgw.buckets rm 'default.775634629.1__multipart_SQL >> Backups/ALL-POND-LIVE_backup_2021_05_26_204508_8473183.d20210526-u200953.bak.s26895803904.zip.0e6LO9b4w9H3HepY-3IW_JSOaysLdFs.1_92' >> >> error removing .rgw.buckets>default.775634629.1__shadow_SQL >> Backups/ALL-POND-LIVE_backup_2021_05_26_204508_8473183.d20210526-u200953.bak.s26895803904.zip.0e6LO9b4w9H3HepY-3IW_JSOaysLdFs.1_92: >> (2) No such file or directory >> >> Checking the presence of the object with the rados tool shows that the object is >> there. >> >> $ cat orphan-list-20230103105849.out |grep -a JSOaysLdFs |grep -a 92 >> default.775634629.1__shadow_SQL >> Backups/ALL-POND-LIVE_backup_2021_05_26_204508_8473183.d20210526-u200953.bak.s26895803904.zip.0e6LO9b4w9H3HepY-3IW_JSOaysLdFs.1_92 >> >> $ cat rados-20230103105849.intermediate |grep -a JSOaysLdFs |grep -a 92 >> default.775634629.1__shadow_SQL >> Backups/ALL-POND-LIVE_backup_2021_05_26_204508_8473183.d20210526-u200953.bak.s26895803904.zip.0e6LO9b4w9H3HepY-3IW_JSOaysLdFs.1_92 >> >> >> Why can't I remove it? I have around 2m objects which can't be removed. What can >> I do to remove them? >> >> Thanks >> >> Andrei >> _______________________________________________ >> ceph-users mailing list -- ceph-users@xxxxxxx >> To unsubscribe send an email to ceph-users-leave@xxxxxxx > > > > -- > Die Selbsthilfegruppe "UTF-8-Probleme" trifft sich diesmal abweichend > im groüen Saal. > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |