There should not be any issues using rgw for other buckets while re-sharding. As for doubling number of objects after reshard is an interesting situation. After the manual reshard is done, there might be leftover from the old bucket index. As during reshard new .dir.new_bucket_index objects are created. They contain all data related to the objects which are stored in buckets.data pool. Just wondering if the issue with the doubled number of objects was related to old bucket index. If so its save to delete old bucket index. In the perfect world, it would be ideal to know the eventoal number of objects inside the bucket and set number of shards to the corresponding setting initially. In the real world when the client re-purpose the usage of the bucket, we have to deal with reshards. пт, 5 лист. 2021, 14:43 користувач mhnx <morphinwithyou@xxxxxxxxx> пише: > I also use this method and I hate it. > > Stopping all of the RGW clients is never an option! It shouldn't be. > Sharding is hell. I was have 250M objects in a bucket and reshard failed > after 2days and object count doubled somehow! 2 days of downtime is not an > option. > > I wonder if I stop the write-read on a bucket and while resharding it is > there any problem of using RGW's with all other buckets? > > Nowadays I advise splitting buckets as much as you can! That means changing > your apps directory tree but this design requires it. > You need to plan object count at least for 5 years and create ones. > Usually I use 101 shards which means 10.100.000 objects. > Also If I need to use versioning I use 2x101 or 3x101 because versions are > hard to predict. You need to predict how many versions you need and set a > lifecycle even before using the bucket! > The max shard that I use 1999. I'm not happy about it but sometimes you > gotta do what you need to do. > Fighting with customers is not an option, you can only advise changing > their apps folder tree but I've never seen someone accept the deal without > arguing. > > My offers usually like this: > 1- Core files bucket: no need to change or very limited changes. "calculate > the object count and multiply with 2" > 2- Hot data bucket: There will be daily changes and versioning. "calculate > the object count and multiply with 3" > 3- Cold data bucket[s]: There will be no daily changes. You should open new > buckets every Year or Month. This is good to keep it clean and steady. No > need for versioning and Multisite Will not suffer due to barely changes. > 4- Temp files bucket[s]: This is so important. If you're crawling millions > of millions objects everyday and delete it at the end of the week or month > then you should definitely use a temp bucket. No versioning, No multisite, > No index if it's possible. > > > > Szabo, Istvan (Agoda) <Istvan.Szabo@xxxxxxxxx>, 5 Kas 2021 Cum, 12:30 > tarihinde şunu yazdı: > > > You mean prepare or reshard? > > Prepare: > > I collect as much information for the users before onboarding so I can > > prepare for their use case in the future and set things up. > > > > Preshard: > > After created the bucket: > > radosgw-admin bucket reshard --bucket=ex-bucket --num-shards=101 > > > > Also when you shard the buckets, you need to use prime numbers. > > > > Istvan Szabo > > Senior Infrastructure Engineer > > --------------------------------------------------- > > Agoda Services Co., Ltd. > > e: istvan.szabo@xxxxxxxxx<mailto:istvan.szabo@xxxxxxxxx> > > --------------------------------------------------- > > > > From: Boris Behrens <bb@xxxxxxxxx> > > Sent: Friday, November 5, 2021 4:22 PM > > To: Szabo, Istvan (Agoda) <Istvan.Szabo@xxxxxxxxx>; ceph-users@xxxxxxx > > Subject: Re: large bucket index in multisite environement > > (how to deal with large omap objects warning)? > > > > Email received from the internet. If in doubt, don't click any link nor > > open any attachment ! > > ________________________________ > > Cheers Istvan, > > > > how do you do this? > > > > Am Do., 4. Nov. 2021 um 19:45 Uhr schrieb Szabo, Istvan (Agoda) < > > Istvan.Szabo@xxxxxxxxx<mailto:Istvan.Szabo@xxxxxxxxx>>: > > This one you need to prepare, you beed to preshard the bucket which you > > know that will hold more than millions of objects. > > > > I have a bucket where we store 1.2 billions of objects with 24xxx shard. > > No omap issue. > > Istvan Szabo > > Senior Infrastructure Engineer > > --------------------------------------------------- > > Agoda Services Co., Ltd. > > e: istvan.szabo@xxxxxxxxx<mailto:istvan.szabo@xxxxxxxxx> > > --------------------------------------------------- > > > > > > > > -- > > Die Selbsthilfegruppe "UTF-8-Probleme" trifft sich diesmal abweichend im > > groüen Saal. > > _______________________________________________ > > ceph-users mailing list -- ceph-users@xxxxxxx > > To unsubscribe send an email to ceph-users-leave@xxxxxxx > > > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |