RGW listing millions of objects takes too much time

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Dear All, 

I have almost 30 million objects and I want to list them and index them somewhere else, 
Im using boto3 with continuation Marker but it takes almost 9 hours

can I run it in multiple threads to make it faster? what solution do you suggest to speedup this process, 


Thanks 

_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux