IPoIB is done with broadcast packets on the Infiniband fabric. Most switches and opensm (by default) setup a broadcast group at the lowest IB speed (SDR), to support all possible IB connections. If you're using pure DDR, you may need to tune the broadcast group in your subnet manager to set the speed to DDR. -- Adam On Thu, Jun 9, 2016 at 3:25 AM, Gandalf Corvotempesta <gandalf.corvotempesta@xxxxxxxxx> wrote: > 2016-06-09 10:18 GMT+02:00 Christian Balzer <chibi@xxxxxxx>: >> IPoIB is about half the speed of your IB layer, yes. > > Ok, so it's normal. I've seen benchmarks on net stating that IPoIB on > DDR should reach about 16-17Gb/s > I'll plan to move to QDR > >> And bandwidth is (usually) not the biggest issue, latency is. > > I've never checked the latency that I had. > Even with IPoIB should be lower than 10GBASE-T > >> Google for "Accelio Ceph". > > Thanks. > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |