Re: Deleting millions of objects

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Rok,

try this:


rgw_delete_multi_obj_max_num - Max number of objects in a single multi-object delete request
  (int, advanced)
  Default: 1000
  Can update at runtime: true
  Services: [rgw]


config set <who> <name> <value>


WHO: client.<rgw-name> or client.rgw

KEY: rgw_delete_multi_obj_max_num

VALUE: 10000

Regards, Joachim

___________________________________
ceph ambassador DACH
ceph consultant since 2012

Clyso GmbH - Premier Ceph Foundation Member

https://www.clyso.com/

Am 17.05.23 um 14:24 schrieb Rok Jaklič:
thx.

I tried with:
ceph config set mon rgw_delete_multi_obj_max_num 10000
ceph config set client rgw_delete_multi_obj_max_num 10000
ceph config set global rgw_delete_multi_obj_max_num 10000

but still only 1000 objects get deleted.

Is the target something different?

On Wed, May 17, 2023 at 11:58 AM Robert Hish <robert.hish@xxxxxxxxxxxx>
wrote:

I think this is capped at 1000 by the config setting. Ive used the aws
and s3cmd clients to delete more than 1000 objects at a time and it
works even with the config setting capped at 1000. But it is a bit slow.

#> ceph config help rgw_delete_multi_obj_max_num

rgw_delete_multi_obj_max_num - Max number of objects in a single multi-
object delete request
   (int, advanced)
   Default: 1000
   Can update at runtime: true
   Services: [rgw]

On Wed, 2023-05-17 at 10:51 +0200, Rok Jaklič wrote:
Hi,

I would like to delete millions of objects in RGW instance with:
mc rm --recursive --force ceph/archive/veeam

but it seems it allows only 1000 (or 1002 exactly) removals per
command.

How can I delete/remove all objects with some prefix?

Kind regards,
Rok
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx

_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx

_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux