On Thu, Aug 22, 2013 at 7:11 AM, Dominik Mostowiec <dominikmostowiec@xxxxxxxxx> wrote: > Hi, > I think about sharding s3 buckets in CEPH cluster, create > bucket-per-XX (256 buckets) or even bucket-per-XXX (4096 buckets) > where XXX is sign from object md5 url. > Could this be the problem? (performance, or some limits) > The two issues that I can think of. One is that there's usually a 1000 buckets per user limitation. This can be easily modified though. The second issue is that you might end up with a huge number of buckets per user, and at that point listing buckets may just take too long. We've seen in the past cases where listing large number of buckets (> 500k) took more than the client timeout period, which in turn retried and retried, which snowballed into a very high load on the gateway (as the original requests were still processing internally). However, this might not be an issue anymore, I do remember we had a problem with streamlining the list bucket responses which I think is already fixed. In any case, if you're not planning to list all the user's buckets then this is moot. Yehuda _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |