Hello. I'm trying to delete buckets but the object deletion is extremely slow if I compare with put operations. I use the command for delete operation: "radosgw-admin bucket rm --bucket=<bucket name> --bypass-gc --purge-objects" My cluster has 190 HDD for EC pool and 30 SSD for index. Ceph version = 14.2.16 Kernel version = 5.4.85-1-lts I've 3 buckets and these objects took 3 days to PUT them in. - bucket 1 = 316.852.681 - bucket 2 = 63.506.363 - bucket 3 = 26.296.788 Total = 406.655.832 objects I started a delete operation job 2 weeks ago and deleted object count is = 85.540.128 So I need 8-10 weeks to delete these objects! WHAT? I put them in 3 days! When I check object deletion speed on a bucket I see 200-500 objects per minute. I use the following command to check speed: "radosgw-admin bucket stats --bucket=<bucket name> | grep num_objects; sleep 60; radosgw-admin bucket stats --bucket=<bucket name> | grep num_objects" Object deletion is faster on small buckets. I've seen 2000-4000 Objects/minute sometimes. Is this because of the listing issue? If so, I can list faster than 1000 objects / second. It doesn't make sense. What is the difference between small and big buckets for delete operation? In the end object deletion is very slow. 1- Is there anything I can do ? 2- Can I start parallel the delete command on different rgw-gateways? What will happen if I do? 3- Is there any improvement with the newer versions of ceph? *One last thing! *I need space and I don't have 8 weeks for this operation! I don't want to break things between RADOS and RGW but I wonder If I delete these objects in RADOS without RGW and then delete bucket index from RGW to prevent "Missing Object Errors" and run the bucket delete command, what will happen? Have a great day folks. _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |