> Has anybody had any experience with running the network routed down all the way to the host? > Hello Nick, yes at SWITCH.ch we run OSPF unnumbered on the switches and on the hosts. Each server has two NICs and we are able to plug the servers to any port on the fabric and OSFP will make the magic :) This makes simpler the design when you want to expand the datacenter or when you want to add more links to existing servers that need more capacity. Remember to put an higher metric on the ToR-Server links otherwise you might end up with flows going through the servers, and that is not what you want. We use Whitebox switches with Cumulus Linux. Back when we started this project in August 2015 we built ubuntu packages for the Quagga version available open source on the Github page of Cumulus Linux. It took us a bit of work to make that quagga running on Ubuntu, but the support from Cumulus was great to sort out the problems. Our setup is dual stack IPv4 and IPv6. On top of that we run Ceph, using IPv6 only for Ceph traffic. Looks like at SWITCH we were not the only ones with this idea, you can find this page dated March 2016: https://support.cumulusnetworks.com/hc/en-us/articles/216805858-Routing-on-the-Host-An-Introduction Cheers, Saverio _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |