Re: large bucket index in multisite environement (how to deal with large omap objects warning)?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



This one you need to prepare, you beed to preshard the bucket which you know that will hold more than millions of objects.

I have a bucket where we store 1.2 billions of objects with 24xxx shard. No omap issue.

Istvan Szabo
Senior Infrastructure Engineer
---------------------------------------------------
Agoda Services Co., Ltd.
e: istvan.szabo@xxxxxxxxx<mailto:istvan.szabo@xxxxxxxxx>
---------------------------------------------------

On 2021. Nov 4., at 17:57, Boris Behrens <bb@xxxxxxxxx> wrote:

Email received from the internet. If in doubt, don't click any link nor open any attachment !
________________________________

Hi everybody,

we maintain three ceph clusters (2x octopus, 1x nautilus) that use three
zonegroups to sync metadata, without syncing the actual data (only one zone
per zonegroup).

Some customer got buckets with >4m objects in our largest cluster (the
other two a very fresh with close to 0 data in it)

How do I handle that in regards of the "Large OMAP objects" warning?
- Sharding is not an option, because it is a multisite environment (at
least thats what I read everywere)
- Limiting the customers is not a great option, because he already got that
huge amount of files in their buckets
- disabling the warning / increasing the threashold, is IMHO a bad option
(people might have put some thinking in that limit and having 40x the limit
is far off the "just roll with it" threashold)

I really hope that someone does have an answer, or maybe there is some
roadmap which addresses this issue.

Cheers
Boris
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux