On 12/17/2015 05:27 PM, Florian Haas wrote: > Hey Wido, > > On Dec 17, 2015 09:52, "Wido den Hollander" <wido@xxxxxxxx > <mailto:wido@xxxxxxxx>> wrote: >> >> On 12/17/2015 06:29 AM, Ben Hines wrote: >> > >> > >> > On Wed, Dec 16, 2015 at 11:05 AM, Florian Haas <florian@xxxxxxxxxxx > <mailto:florian@xxxxxxxxxxx> >> > <mailto:florian@xxxxxxxxxxx <mailto:florian@xxxxxxxxxxx>>> wrote: >> > >> > Hi Ben & everyone, >> > >> > >> > Ben, you wrote elsewhere >> > > (http://lists.ceph.com/pipermail/ceph-users-ceph.com/2015-August/003955.html) >> > that you found approx. 900k objects to be the threshold where index >> > sharding becomes necessary. Have you found that to be a reasonable >> > rule of thumb, as in "try 1-2 shards per million objects in your > most >> > populous bucket"? Also, do you reckon that beyond that, more shards >> > make things worse? >> > >> > >> > >> > Oh, and to answer this part. I didn't do that much experimentation >> > unfortunately. I actually am using about 24 index shards per bucket >> > currently and we delete each bucket once it hits about a million >> > objects. (it's just a throwaway cache for us) Seems ok, so i stopped >> > tweaking. >> > >> >> I have a use case where I need to store 350 Million objects in a single >> bucket. > > How many OSDs are in that cluster? > 1800 and it will grow towards 2500 in Q1 2016. >> I tested with 4096 shards and that works. Creating the bucket takes a >> few seconds though. > > Does "that works" mean that you have actually uploaded 350M objects into > that one bucket? > No, still in progress. The bucket functions, that is what I meant. > If so, can you give me a feel for your typical object size? > It varies. It is a archiving solution and I'm not in control there. > Also, what's the performance drop you saw in bucket listing, vs. having > fewer shards or no sharding at all? > There is a drop in listing performance, didn't completely measure it, but I think that with 4k shards the listing was a few seconds. In this use-case we are not going to list the bucket, ever. > Cheers, > Florian > -- Wido den Hollander 42on B.V. Ceph trainer and consultant Phone: +31 (0)20 700 9902 Skype: contact42on _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |