On 04/25/2013 07:39 AM, Anders Saaby wrote:
Hi, We are working on prototype infrastructure for RADOS clusters, and are now ready to deploy the first production size storage pool. One question remains; How many placement groups will we need, balancing memory footprint and ability to level data placement and data reads. - And still keeping stuff within sane limits. Our initial plan is to deploy 4PB pools, based on 4TB drives with 3 replicas (One OSD/disk). So, 3.000 disks per pool. Acording to the documentation 1), we should have: 3.000 OSDs * 100 / 3 replicas == 100.000 placement groups. From the maillist, 100.000 PG's is way more than I have seen, so, do you have any insights and advises on pg_num for a RADOS pool with these characteristics? Also, will it be a problem with a pg_num size this bit, if the pool is started out with only ~100 OSDs, and then grown to 3.000.
I pretty regularly test single-mon configurations with 64k PGs. ~100k PGs tends to be starting to get a bit intense, but with a larger mon cluster and some tweaking it should be doable.
I don't mean to push anything on you, but if you guys are really thinking about deploying multiple 4PB pools, you might want to talk to us about some kind of support/consulting agreement. That's a lot of storage!
Mark
Thanks in advance, Anders 1: http://ceph.com/docs/master/rados/operations/placement-groups/ -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
-- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
