On 01/17/2013 07:32 AM, Joseph Glanville wrote:
On 17 January 2013 20:46, Gandalf Corvotempesta
<gandalf.corvotempesta@xxxxxxxxx> wrote:
2013/1/16 Mark Nelson<mark.nelson@xxxxxxxxxxx>:
I don't know if I have to use a single two port IB card (switch
redundancy and no card redundancy) or
I have to use two single port cards. (or a single one port IB?)
On the topic of IB..
But slightly off-topic all the same.. I would love to attempt getting
Ceph running on rsockets if I could find the time (alas we don't run
Ceph).
rsockets is a fully userland implementation of BSD sockets over RDMA,
supporting fork and all the usual goodies, in theory unless you are
using the kernel RBD module (of the kernel FS module etc) you should
be able to run it on rsockets and enjoy a considerable performance
increase.
rsockets is available in the librdmacm git up on Open Fabrics and dev
+ support happens on the linux-rdma list.
There's been some talk about rsockets on the list before. I think there
are a couple of different folks that have tried (succeeded?) in getting
it working. barring that, it sounds like if you tune interrupt affinity
settings and various other bits you can get IPoIB up into the 2GB/s+
range which while not RDMA speed, is at least better than 10GbE.
--
Mark Nelson
Performance Engineer
Inktank
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
