Re: hardware setup recommendations wanted

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Kai,

El 28/8/23 a las 9:35, Kai Zimmer escribió:
Dear listers,

my employer already has a production Ceph cluster running but we need a second one. I just wanted to ask your opininion on the following setup. It is planned for 500 TB net capacity, expandable to 2 PB. I expect the number of OSD servers to double in the next 4 years. Erasure Code 3:2 will be used for OSDs. Usage will be file storage, Rados block devices and S3:

5x OSD servers (12x18 TB Toshiba MG09SCA18TE SAS spinning disks for data, 2x512 GB Samsung PM9A1 M.2 NVME SSD 0,55 DWPD for system, 1xAMD 7313P CPU with 16 cores @3GHz, 256 GB RAM, LSI SAS 9500 HBA, Broadcom P425G network adapter 4x25 Gbit/s)

No SSD/NVMe disks for those spinners?


3x MON servers (1x2 TB Samsung PM9A1 M.2 NVME SSD 0,55 DWPD for system, 2x1.6TB Kioxia CD6-V SSD 3.0 DWPD for data, 2x Broadcom P210/N210 network 4x10 GBit/s, 1xAMD 7232P CPU with 8 cores @3.1 GHz, 64 GB RAM)

3x MDS servers (1x2 TB Samsung PM9A1 M.2 NVME SSD 0,55 DWPD for system, 2x1.6 TB Kioxia CD6-V SSD 3.0 DWPD for data, 2x Broadcom P210/N210 network 4x10 GBit/s, 1xAMD 7313P CPU with 16 cores @3 GHz, 128 GB RAM)

OSD servers will be connected via 2x25 GBit fibre interfaces "backend" to

2x Mikrotik CRS518-16XS-2XQ (which are connected for high-availability via 100 GBit)

For the "frontend" connection to servers/clients via 2x10 GBit we're looking into

3x Mikrotik CRS326-24S+2Q+RM (which are connected for high-availability via 40 GBit)

Our experience with those CR326 has been good. We normally use SwOS instead of RouterOS, which gives you a simple and clean web interface and just the features needed for a switch that will just be a switch :)

Cheers


Especially for the "frontend" switches i'm looking for alternatives. Currently we use Huawei C6810-32T16A4Q-LI models with 2x33 LACP connections connected via 10 GBit/s RJ45. But those had ports blocking after a number of errors which resulted in some trouble. We'd like to avoid IOS and clones in general and would prefer a decent web interface.

Any comments/recommendations?

Best regards,

Kai
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx

Eneko Lacunza
Zuzendari teknikoa | Director técnico
Binovo IT Human Project

Tel. +34 943 569 206 | https://www.binovo.es
Astigarragako Bidea, 2 - 2º izda. Oficina 10-11, 20180 Oiartzun

https://www.youtube.com/user/CANALBINOVO
https://www.linkedin.com/company/37269706/
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux