Hello, this is not an answer to the question directly, but you could consider the following to double bandwidth: * Run each ceph node with two NICs, each has an own IP, e.g. one node has 192.0.2.10/24 and 192.0.2.11/24 * In ceph.conf you bind 50% of the OSDs to each of those IPs: [osd.XY] ... public_addr = ... cluster_addr = 192.0.2.x * With an equally distributed traffic (enough OSD) should nearly double your bandwidth * To get redundancy, you could extend that config using keepalived/vrrp to switchover the IP of a failing NIC/switch to the other one. I am pretty aware that we also have linux bonding with mode slb, but to my experience that didn't work very well with COTS switches, maybe due to ARP learing issues. (We ended up buying Juniper QFX-5100 with MLAG support). Best Regards, Andreas P.S. I didn't try out the setup from above yet. If anyone did already or will do, I would be happy about feedback. On 16.02.21 16:56, Mario Giammarco wrote: > Il giorno lun 15 feb 2021 alle ore 15:16 mj <lists@xxxxxxxxxxxxx> ha > scritto: > >> >> On 2/15/21 1:38 PM, Eneko Lacunza wrote: >>> Do you really need MLAG? (the 2x10G bandwith?). If not, just use 2 >>> simple switches (Mikrotik for example) and in Proxmox use an >>> active-pasive bond, with default interface in all nodes to the same >> switch. >> >> Since we are now on SSD OSDs only, and our aim is to be able to add more >> OSD nodes, yes: I think we should aim for more than 10G bandwidth. >> >> So go for 40G. LACP will not give you 2x10 magical bandwidth doubling. > BTW: I am using mikrotik 10g switches and they have great value. > BTW2: if you use Proxmox you do not need LACP you can use linux round robin > support that has the same performance of LACP and it does not require > switches support. > > >> Thanks! >> _______________________________________________ >> ceph-users mailing list -- ceph-users@xxxxxxx >> To unsubscribe send an email to ceph-users-leave@xxxxxxx >> > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > -- Andreas John net-lab GmbH | Frankfurter Str. 99 | 63067 Offenbach Geschaeftsfuehrer: Andreas John | AG Offenbach, HRB40832 Tel: +49 69 8570033-1 | Fax: -2 | http://www.net-lab.net Facebook: https://www.facebook.com/netlabdotnet Twitter: https://twitter.com/netlabdotnet _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |