The one side effect you might see if you actually create a million or more bucklets for a single user, is a large OMAP warning, as the mapping of buckets to owners is not sharded. Matt On Mon, May 24, 2021 at 1:51 AM Michal Strnad <michal.strnad@xxxxxxxxx> wrote: > > Thank you. So we can create millions of buckets associated to only one > S3 account without any limitation or side effect? Does anyone use it > this way? > > Many thanks in advance. > Michal > > > On 5/23/21 9:42 PM, Janne Johansson wrote: > > Many buckets. > > > > Den sön 23 maj 2021 kl 20:53 skrev Michal Strnad <michal.strnad@xxxxxxxxx>: > >> > >> Hi all, > >> > >> We need to store millions of files using S3 protocol in Ceph (version > >> Nautilus), but have projects where isn't appropriate or possible to > >> create a lot of S3 accounts. Is it better to have multiple S3 buckets or > >> one bucket with sub folders? > >> > >> For example AWS service from Amazon allows you to create up to 100 > >> buckets in each of your AWS cloud accounts. You can request more > >> buckets, up to a maximum quota of 1,000, by submitting a service limit > >> increase. There is no limit on the number of objects you can store in a > >> bucket, but in the Ceph we run into a problem with listing and > >> resharding with a millions of files in one bucket. > >> > >> Thank you > >> > >> Michal > >> > >> _______________________________________________ > >> ceph-users mailing list -- ceph-users@xxxxxxx > >> To unsubscribe send an email to ceph-users-leave@xxxxxxx > > > > > > > > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx -- Matt Benjamin Red Hat, Inc. 315 West Huron Street, Suite 140A Ann Arbor, Michigan 48103 http://www.redhat.com/en/technologies/storage tel. 734-821-5101 fax. 734-769-8938 cel. 734-216-5309 _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |