Try s5cmd k > On 30 Jan 2021, at 17:00, michal.strnad@xxxxxxxxx wrote: > > We are testing our S3 Ceph endpoints and we are not satisfied with its > speed. Our results are something between around 120 - 150 MB/s depending > on small/bigger files. This is good for 1Gbps connection, but not for > 10GE or more. > > We've tried the most recent versions of the AWS CLI, s3cmd, s4cmd, s3fs > ... programs. Of course we are using multipart upload/download which is > precondition for parallel upload/download. Also we tried multi-thread > (25 or more threads) transfer in s4cmd but still we don't get proper > results. > > For proof of concept that high speed can be achieved we have written > small script in bash which uses multi-part & parallel transfer and can > saturate at least 10GE without problem. > > I would like to ask you, if you know proper program and its parameters, > so we can saturate n x 10GE if needed? > > We are using the latest nautilus. > S3 gateways have much more computer power and bandwidth to internet then > it is used right now. _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |