AFAIK the standing recommendation for all flash setups is to prefer fewer but faster cores, so something like a 75F3 might be yielding better latency. Plus you probably want to experiment with partitioning the NVMEs and running multiple OSDs per drive - either 2 or 4. On Sat, 7 Oct 2023 at 08:23, Gustavo Fahnle <gfahnle@xxxxxxxxxxx> wrote: > Hi, > > Currently, I have an OpenStack installation with a Ceph cluster consisting > of 4 servers for OSD, each with 16TB SATA HDDs. My intention is to add a > second, independent Ceph cluster to provide faster disks for OpenStack VMs. > The idea for this second cluster is to exclusively provide RBD services to > OpenStack. I plan to start with a cluster composed of 3 mon/mgr nodes > similar to what we currently have (3 virtualized servers with VMware) with > 4 cores, 8GB of memory, 80GB disk and 10GB network > each server. > In the current cluster, these nodes have low resource consumption, less > than 10% CPU usage, 40% memory usage, and less than 100Mb/s of network > usage. > > For the OSDs, I'm thinking of starting with 3 or 4 servers, specifically > Supermicro AS-1114S-WN10RT, each with: > > 1 AMD EPYC 7713P Gen 3 processor (64 Core, 128 Threads, 2.0GHz) > 256GB of RAM > 2 x NVME 1TB for the operating system > 10 x NVME Kingston DC1500M U.2 7.68TB for the OSDs > Two Intel NIC E810-XXVDA2 25GbE Dual Port (2 x SFP28) PCIe 4.0 x8 cards > Connected to 2 MikroTik CRS518-16XS-2XQ-RM switches at 100GbE per server > Connection to OpenStack would be via 4 x 10GB to our core switch. > > I would like to hear opinions about this configuration, recommendations, > criticisms, etc. > > If any of you have references or experience with any of the components in > this initial configuration, they would be very welcome. > > Thank you very much in advance. > > Gustavo Fahnle > > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |