Re: Infiniband backend OSD communication

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Ok, so ipoib is required...

><>
nathan stratton


On Mon, Jan 6, 2020 at 4:45 AM Wei Zhao <zhao6305@xxxxxxxxx> wrote:
From my understanding, the basic idea is that ceph exchange rdma
information(qp,gid and so) through ip address on rdma device, and then
communicate with each other throng rdma. But in my tests,  there
seemed to be some issues in that codes.

On Fri, Jan 3, 2020 at 2:24 AM Nathan Stratton <nathan@xxxxxxxxxxxx> wrote:
>
> I am working on upgrading my current ethernet only ceph cluster to a combined ethernet frontend and infiniband backend. From my research I understand that I set:
>
> ms_cluster_type = async+rdma
> ms_async_rdma_device_name = mlx4_0
>
> What I don't understand is how does ceph know how to reach each OSD over RDMA? Do I have to run IPoIB on top of infiniband and use that for OSD addresses?
>
> Is there a way to use infiniband on backend without IPoIB and just use rdma verbs?
>
> ><>
> nathan stratton
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux