On Wed, 2019-03-06 at 22:14 +-0200, Yuval Shaia wrote: +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 Suggestion: To enhance 'loopback' performances, can you consider +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 using shared memory or any other IPC instead of going thought +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 the +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 network stack? +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 Loopback driver in this patchset doesn't use network stack. +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 It is just 2000 lines of wrapper to memcpy() to enables +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 applications to use +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 rdma. +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 To have a dedicated driver just for the loopback will force the user +AD4 +AD4 +AD4 +AD4 +AD4 to do a smart select, i.e. to use lo device for local traffic and rxe for non- +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 local. +AD4 +AD4 +AD4 +AD4 No. when application is written using rdmacm, everything works based on +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 the ip address. +AD4 +AD4 +AD4 +AD4 It will pick the right rdma device that matches this ip. +AD4 +AD4 +AD4 +AD4 It would be 'lo' when connections are on 127.0.0.1. +AD4 +AD4 +AD4 +AD4 When application such as MPI, will have to anyway specify the which rdma +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 device they want to use in system. +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 But what if one wants to stay at the verb level and not use rdmacm API? +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 +AD4 Sure. He can stay at verb level where he anyway have to explicitly give the device name. +AD4 +AD4 And that's is exactly the problem+ACE +AD4 +AD4 With qemu, the ibdev is given at the command-line of the virtual machine so +AD4 if two guests starts on the same host it is ok to give them the lo device +AD4 as backend but what will happen when one of the VMs will migrate to another +AD4 host? The traffic will break since the lo device cannot go outside. Hi Yuval, I think what you are describing falls outside the use cases Parav has in mind. I think that optimizing RDMA over loopback, even if that loopback only works inside a single VM, is useful. Bart.
