Mimic (and higher) contain a new async gc mechanism, which should handle this workload internally. Matt On Mon, Apr 23, 2018 at 2:55 PM, David Turner <drakonstein@xxxxxxxxx> wrote: > When figuring out why space is not freeing up after deleting buckets and > objects in RGW, look towards the RGW Garbage Collection. This has come up > on the ML several times in the past. I am almost finished catching up on a > GC of 200 Million objects that was taking up a substantial amount of space > in my cluster. I did this by running about 30 screens with the command > `while true; do radosgw-admin gc process; sleep 10; done` in each of them. > It appears that there are 32 available sockets for the gc to be processed > and this helped us catch up on 200M objects in under 2 months. > > On Mon, Apr 16, 2018 at 12:01 PM Robert Stanford <rstanford8896@xxxxxxxxx> > wrote: >> >> >> This doesn't work for me: >> >> for i in `radosgw-admin bucket list`; do radosgw-admin bucket unlink >> --bucket=$i --uid=myuser; done (tried with and without '=') >> >> Errors for each bucket: >> >> failure: (2) No such file or directory2018-04-16 15:37:54.022423 >> 7f7c250fbc80 0 could not get bucket info for bucket="bucket5", >> >> On Mon, Apr 16, 2018 at 8:30 AM, Casey Bodley <cbodley@xxxxxxxxxx> wrote: >>> >>> >>> >>> On 04/14/2018 12:54 PM, Robert Stanford wrote: >>> >>> >>> I deleted my default.rgw.buckets.data and default.rgw.buckets.index >>> pools in an attempt to clean them out. I brought this up on the list and >>> received replies telling me essentially, "You shouldn't do that." There was >>> however no helpful advice on recovering. >>> >>> When I run 'radosgw-admin bucket list' I get a list of all my old >>> buckets (I thought they'd be cleaned out when I deleted and recreated >>> default.rgw.buckets.index, but I was wrong.) Deleting them with s3cmd and >>> radosgw-admin does nothing; they still appear (though s3cmd will give a >>> '404' error.) Running radosgw-admin with 'bucket check' and '--fix' does >>> nothing as well. So, how do I get myself out of this mess. >>> >>> On another, semi-related note, I've been deleting (existing) buckets and >>> their contents with s3cmd (and --recursive); the space is never freed from >>> ceph and the bucket still appears in s3cmd ls. Looks like my radosgw has >>> several issues, maybe all related to deleting and recreating the pools. >>> >>> Thanks >>> >>> >>> _______________________________________________ >>> ceph-users mailing list >>> ceph-users@xxxxxxxxxxxxxx >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>> >>> >>> The 'bucket list' command takes a user and prints the list of buckets >>> they own - this list is read from the user object itself. You can remove >>> these entries with the 'bucket unlink' command. >>> >>> _______________________________________________ >>> ceph-users mailing list >>> ceph-users@xxxxxxxxxxxxxx >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>> >> >> _______________________________________________ >> ceph-users mailing list >> ceph-users@xxxxxxxxxxxxxx >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- Matt Benjamin Red Hat, Inc. 315 West Huron Street, Suite 140A Ann Arbor, Michigan 48103 http://www.redhat.com/en/technologies/storage tel. 734-821-5101 fax. 734-769-8938 cel. 734-216-5309 _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |