On Fri, Apr 22, 2022 at 3:20 PM Cory Snyder <csnyder@xxxxxxxxx> wrote: > > Hi all, > > Does anyone have any guidance on the maximum number of bucket index shards > to optimize performance for buckets with a huge number of objects? It seems > like there is probably a threshold where performance starts to decrease > with an increased number of shards (particularly bucket listings). More > specifically, if I have N OSDs in the bucket index pool, does it make sense > to allow a bucket to have more than N index shards? with respect to write parallelism, i think the most interesting limit is the PG count of the index pool. my understanding is that the OSDs can only handle a single write at a time per PG due to the rados recovery model. so you'd expect to see index write performance increase as you raise the shard count, but level off as you get closer to that PG count > Perhaps some multiple > of N makes sense, with the value of the multiplier influenced by > osd_op_num_threads_per_shard and osd_op_num_shards? i'm less familiar with these OSD configurables, but it's possible that they'd impose limits on parallelism below the PG count > > Thanks in advance for any theoretical or empirical insights! if you need to list these huge buckets, you'll want to strike a balance between write parallelism and the latency of bucket listing requests. once that request latency reaches the client's retry timer, you'll really start to see listing performance fall off a cliff > > Cory Snyder > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |