I can think of 1 vendor who has made some of the compromises that you talk of although memory and CPU is not one of them they are limited on slots and NVME capacity. But there are plenty of other vendors out there who use the same model of motherboard across the whole chassis range so there isn’t a compromise in terms of slots and CPU. The compromise may come with the size of the chassis in that a lot of these bigger chassis can also be deeper to get rid of the compromises. The reality with an OSD node is you don't need that many slots or network ports. From: Janne Johansson <icepic.dz@xxxxxxxxx> Date: Thursday, 23 April 2020 at 08:08 To: Darren Soothill <darren.soothill@xxxxxxxx> Cc: ceph-users@xxxxxxx <ceph-users@xxxxxxx> Subject: Re: Re: Dear Abby: Why Is Architecting CEPH So Hard? Den tors 23 apr. 2020 kl 08:49 skrev Darren Soothill <darren.soothill@xxxxxxxx<mailto:darren.soothill@xxxxxxxx>>: If you want the lowest cost per TB then you will be going with larger nodes in your cluster but it does mean you minimum cluster size is going to be many PB’s in size. Now the question is what is the tax that a particular chassis vendor is charging you. I know from the configs we do on a regular basis that a 60 drive chassis will give you the lowest cost per TB. BUT it has implications. Your cluster size needs to be up in the order of 10PB minimum. 60 x 18TB gives you around 1PB per node. Oh did you notice here we are going for the bigger disk drives. Why because the more data you can spread your fixed costs across the lower the overall cost per GB. I don't know all models, but the computers I've looked at with 60 drive slots will have a small and "crappy" motherboard, with few options, not many buses/slots/network ports and low amounts of cores, DIMM sockets and so on, counting on you to make almost a passive storage node on it. I have a hard time thinking the 60*18TB OSD recovery requirements in cpu and ram would be covered in any way by the kinds of 60-slot boxes I've seen. Not that I focus on that area, but it seems like a common tradeoff, Heavy Duty(tm) motherboards or tons of drives. -- May the most significant bit of your life be positive. _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |