Re: How to organize data in S3

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Thank you. So we can create millions of buckets associated to only one
S3 account without any limitation or side effect? Does anyone use it
this way?

Many thanks in advance.
Michal


On 5/23/21 9:42 PM, Janne Johansson wrote:
> Many buckets.
> 
> Den sön 23 maj 2021 kl 20:53 skrev Michal Strnad <michal.strnad@xxxxxxxxx>:
>>
>> Hi all,
>>
>> We need to store millions of files using S3 protocol in Ceph (version
>> Nautilus), but have projects where isn't appropriate or possible to
>> create a lot of S3 accounts. Is it better to have multiple S3 buckets or
>> one bucket with sub folders?
>>
>> For example AWS service from Amazon allows you to create up to 100
>> buckets in each of your AWS cloud accounts. You can request more
>> buckets, up to a maximum quota of 1,000, by submitting a service limit
>> increase. There is no limit on the number of objects you can store in a
>> bucket, but in the Ceph we run into a problem with listing and
>> resharding with a millions of files in one bucket.
>>
>> Thank you
>>
>> Michal
>>
>> _______________________________________________
>> ceph-users mailing list -- ceph-users@xxxxxxx
>> To unsubscribe send an email to ceph-users-leave@xxxxxxx
> 
> 
> 

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature

_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux