For what it worth, yahoo published their setup some years ago: https://yahooeng.tumblr.com/post/116391291701/yahoo-cloud-object-store-object-storage-at 54 nodes per cluster for 3.2PB of raw storage, I guess this leads to 16 * 4TB hdd per node, thus 896 per cluster (they may have used ssd as journal, anyway) This setup is only viable because they are using HTTP, which load is easily and highly balanceable On 05/16/2018 06:20 PM, Matthew Vernon wrote: > Hi, > > On 14/05/18 17:49, Marc Boisis wrote: > >> Currently we have a 294 OSD (21 hosts/3 racks) cluster with RBD clients >> only, 1 single pool (size=3). > > That's not a large cluster. > >> We want to divide this cluster into several to minimize the risk in case >> of failure/crash. >> For example, a cluster for the mail, another for the file servers, a >> test cluster ... >> Do you think it's a good idea ? > > I'd venture the opinion that you cluster isn't yet big enough to be > thinking about that; you get increased reliability with a larger cluster > (each disk failure is a smaller % of the whole, for example); our > largest cluster here is 3060 OSDs... > > We've grown this from a start of 540 OSDs. > >> Do you have experience feedback on multiple clusters in production on >> the same hardware: > > I think if you did want to have multiple clusters, you'd want to have > each cluster on different hardware. > > Regards, > > Matthew > > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |