|
If you want to build up with Viatta. And this give you the possibility to have a fully feature OS. What kind of hardware would you use to build up a switch? Il 08/10/2014 09:10, Christian Balzer
ha scritto:
On Wed, 08 Oct 2014 00:45:06 +0000 Scott Laird wrote:IIRC, one thing to look out for is that there are two ways to do IP over Infiniband. You can either do IP over Infiniband directly (IPoIB), or encapsulate Ethernet in Infiniband (EoIB), and then do IP over the fake Ethernet network. IPoIB is more common, but I'd assume that IB<->Ethernet bridges really only bridge EoIB.Most of them do indeed, alas the 4036E supposedly has a FPGA based IPoIB to Ethernet gateway. Probably another reason to build your own, aside from the price tag. ^o^ ChristianOn Tue Oct 07 2014 at 5:34:57 PM Christian Balzer <chibi@xxxxxxx> wrote:On Tue, 07 Oct 2014 20:40:31 +0000 Scott Laird wrote:I've done this two ways in the past. Either I'll give each machine an Infiniband network link and a 1000baseT link and use the Infiniband one as the private network for Ceph, or I'll throw an Infiniband card into a PC and run something like Vyatta/VyOS on it and make it a router, so IP traffic can get out of the IB network. Of course, those have both been for test labs. YMMV.That. Of course in a production environment you would want something with 2 routers in a failover configuration. And there are switches/gateways that combine IB and Ethernet, but they tend to be not so cheap. ^^ More below.On Tue Oct 07 2014 at 11:05:23 AM Massimiliano Cuttini <max@xxxxxxxxxxxxx> wrote:Hi Christian, When you say "10 gig infiniband", do you mean QDRx4 Infiniband (usually flogged as 40Gb/s even though it is 32Gb/s, but who's counting), which tends to be the same basic hardware as the 10Gb/s Ethernet offerings from Mellanox? A brand new 18 port switch of that caliber will only cost about 180$ per port, too. I investigate about infiniband but i didn't found affordable prices at all.Then you're doing it wrong or comparing apples to oranges (you of course need to compare IB switches to similar 10GbE ones). And the prices of HCA (aka network cards in the servers) and cabling.Moreover how do you connect your *l**egacy node servers* to your *brand new storages* if you have Infiniband only on storages & switches? Is there any mixed switch that allow you both to connect with Infiniband and Ethernet? If there is, please send specs because i cannot find just by google it.The moment you type in "infiniband et" google will already predict amongst other pertinent things "infiniband ethernet gateway" and "infiniband ethernet bridge". But even "infiniband ethernet switch" has a link telling you pretty much what was said here now at the 6th position: http://www.tomshardware.com/forum/44997-42-connect- infiniband-switch-ethernet ChristianThanks, Max _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com-- Christian Balzer Network/Systems Engineer chibi@xxxxxxx Global OnLine Japan/Fusion Communications http://www.gol.com/ _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com |
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
