On 04/18/2013 03:40 PM, Gandalf Corvotempesta wrote:
Hi,
will RDMA be supported in the shortterm?
I'm planning an infrastructure and I don't know if starting with IB
QDR or 10GbE.
Depends on your definition of RDMA, supported, and short term. ;)
We like the idea of using rsockets as it would be somewhat easier to
implement and it looks like it may get pretty close to RDMA levels of
throughput and latency. Having said that, it's young and there's not
currently a kernel implementation. We'll probably spend some time
evaluating rsockets before we'll dive into an RDMA based messenger entirely.
IB is much cheaper than 10GbE and with RDMA should be 4x faster, but
with IPoIB as workaround I've read that is very very heavy on CPU and
very slow (15gbit more or less)
I've done testing with IPoIB on QDR IB and was able to get around 2GB/s.
You may be able to get more throughput with interrupt affinity tuning,
but I can at least say 2GB/s is possible. I didn't look closely at CPU
overhead. Having said that, 10GbE can use some CPU too...
What do you suggest? 10GbE or IB QDR for the future?
10GbE is fully supported and widely used with Ceph while IB is a bit
more complicated with fewer users. Having said that, IPoIB seems to
work just fine, and there is potential in the future for even better
performance. Which one is right for you probably depends on the
existing network infrastructure you are using, how fast your OSD nodes
are, and what you are trying to do. Sadly there is no easy answer. :)
Mark
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
