On Wed, Mar 06, 2019 at 04:40:34PM +0000, Parav Pandit wrote: > > > > -----Original Message----- > > From: Yuval Shaia <yuval.shaia@xxxxxxxxxx> > > Sent: Wednesday, March 6, 2019 1:43 AM > > To: Parav Pandit <parav@xxxxxxxxxxxx> > > Cc: Ira Weiny <ira.weiny@xxxxxxxxx>; Leon Romanovsky <leon@xxxxxxxxxx>; > > Dennis Dalessandro <dennis.dalessandro@xxxxxxxxx>; bvanassche@xxxxxxx; > > linux-rdma@xxxxxxxxxxxxxxx; Marcel Apfelbaum > > <marcel.apfelbaum@xxxxxxxxx>; Kamal Heib <kheib@xxxxxxxxxx> > > Subject: Re: [EXPERIMENTAL v1 0/4] RDMA loopback device > > > > > > > > > > > > > Host is not aware of the IP stack of the VM. > > > > > How do you resolve mac address for the destination IP used in > > > > > guest VM in > > > > host, and how do you program right source mac and destination mac of > > > > the QP in host using ibverbs client? > > > > > I was asking Aviad in Mellanox to use devx interface do have > > > > > passthrough > > > > programming. > > > > > So want to understand how are you doing this QEMU as its close to > > > > production now? > > > > > > > > Not sure i fully understand your question, ex why host needs to know > > > > the guest IP. > > > > Anyway, the flow is like this: > > > > - In guest, ib_core calls the driver's add_gid hook when gid entry is > > > > created > > > > - Driver in guest passes binding info to qemu device (sgid and gid) > > > > - qemu device adds this gid to host gid table (via netlink or QMP), note > > > > that sgid in host probably is different. Please also note that at this > > > > stage we have this gid defined twice in the fabric but since one is in > > > > guest, which is hidden, we are ok. (1098) > > > > - When guest creates QP it passes the guest sgid to qemu device which > > > > replace it with the host sgid. (#436) > > > > > > > > Since gid is defined in the host the all the routing is done as usual. > > > > > > > > Full details of the above is available here > > > > > > https://github.com/qemu/qemu/commit/2b05705dc8ad80c09a3aa9cc70c14f > > > > b8323b0fd3 > > > > > > > > Hope this answers your question. > > > > > > > I took cursory look. I am almost sure that if guest VM's IP address is not > > added to host, modify_qp() in kernel is going to fail. Unless you use a > > different GID in host. And build some smart way to figure out which one to > > use. > > > > The option to use different GID is too complex and even not applicable as > > you have to make the fabric know to route traffic to it. > > Check the above step #3, the guest GID (and so also IP) is pops up to the > > host level. > > > Interesting. How do you know what subnet mask to use when you add IP address in host? Sorry, my bad, forgot to mentioned it - the subnet_prefix is also provided by the guest driver.
