Den tis 24 aug. 2021 kl 09:46 skrev Francesco Piraneo G. <fpiraneo@xxxxxxxxxxx>: > Il 24.08.21 09:32, Janne Johansson ha scritto: > >> As a simple test I copied an Ubuntu /usr/share/doc (580 MB in 23'000 files): > >> - rsync -a to a Cephfs took 2 min > >> - s3cmd put --recursive took over 70 min > >> Users reported that the S3 access is generally slow, not only with s3tools. > > Single per-object accesses and writes on S3 are slower, since they > > involve both client and server side checksumming, a lot of http(s) > > stuff before the actual operations start and I don't think there is a > > lot of connection reuse or pipelining being done so you are going to > > make some 23k requests, each taking a non-zero time to complete. > > > Question: Is Swift compatible protocol faster? Probably not, but make a few tests and find out how it works at your place. It's kind of easy to rig both at the same time, so you can test on exactly the same setup. > Use case: I have to store indefinite files quantity for a data storage > service; I thought object storage is the unique solution; each file is > identified by UUID, no metadata on file, files are chunked 4Mb size each. That sounds like a better case for S3/Swift. > In such case cephfs is the best suitable choice? One factor to add might be "will it be reachable from the outside?", since radosgw is kind of easy to put behind a set of load balancers, that can wash/clean incoming traffic and handle TLS offload and things like that. Putting cephfs out on the internet might have other cons. -- May the most significant bit of your life be positive. _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |