Hey Wido,
On Dec 17, 2015 09:52, "Wido den Hollander" <wido@xxxxxxxx> wrote:
>
> On 12/17/2015 06:29 AM, Ben Hines wrote:
> >
> >
> > On Wed, Dec 16, 2015 at 11:05 AM, Florian Haas <florian@xxxxxxxxxxx
> > <mailto:florian@xxxxxxxxxxx>> wrote:
> >
> > Hi Ben & everyone,
> >
> >
> > Ben, you wrote elsewhere
> > (http://lists.ceph.com/pipermail/ceph-users-ceph.com/2015-August/003955.html)
> > that you found approx. 900k objects to be the threshold where index
> > sharding becomes necessary. Have you found that to be a reasonable
> > rule of thumb, as in "try 1-2 shards per million objects in your most
> > populous bucket"? Also, do you reckon that beyond that, more shards
> > make things worse?
> >
> >
> >
> > Oh, and to answer this part. I didn't do that much experimentation
> > unfortunately. I actually am using about 24 index shards per bucket
> > currently and we delete each bucket once it hits about a million
> > objects. (it's just a throwaway cache for us) Seems ok, so i stopped
> > tweaking.
> >
>
> I have a use case where I need to store 350 Million objects in a single
> bucket.
How many OSDs are in that cluster?
> I tested with 4096 shards and that works. Creating the bucket takes a
> few seconds though.
Does "that works" mean that you have actually uploaded 350M objects into that one bucket?
If so, can you give me a feel for your typical object size?
Also, what's the performance drop you saw in bucket listing, vs. having fewer shards or no sharding at all?
Cheers,
Florian
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
