On Wed, Mar 06, 2019 at 12:39:17PM -0800, Bart Van Assche wrote: > On Wed, 2019-03-06 at 22:14 +0200, Yuval Shaia wrote: > > > > > > > > > > > > > > > > > > > > > > Suggestion: To enhance 'loopback' performances, can you consider > > > > > > > > using shared memory or any other IPC instead of going thought > > > > > > > > the > > > > > > > > > > > > network stack? > > > > > > > > > > > > > > > > > > > > > > Loopback driver in this patchset doesn't use network stack. > > > > > > > It is just 2000 lines of wrapper to memcpy() to enables > > > > > > > applications to use > > > > > > > > > > > > rdma. > > > > > > > > > > > > To have a dedicated driver just for the loopback will force the user > > > > > > to do a smart select, i.e. to use lo device for local traffic and rxe for non- > > > > > > > > local. > > > > > No. when application is written using rdmacm, everything works based on > > > > > > > > the ip address. > > > > > It will pick the right rdma device that matches this ip. > > > > > It would be 'lo' when connections are on 127.0.0.1. > > > > > When application such as MPI, will have to anyway specify the which rdma > > > > > > > > device they want to use in system. > > > > > > > > But what if one wants to stay at the verb level and not use rdmacm API? > > > > > > > > > > Sure. He can stay at verb level where he anyway have to explicitly give the device name. > > > > And that's is exactly the problem! > > > > With qemu, the ibdev is given at the command-line of the virtual machine so > > if two guests starts on the same host it is ok to give them the lo device > > as backend but what will happen when one of the VMs will migrate to another > > host? The traffic will break since the lo device cannot go outside. > > Hi Yuval, > > I think what you are describing falls outside the use cases Parav has in mind. I > think that optimizing RDMA over loopback, even if that loopback only works inside > a single VM, is useful. > > Bart. Sure, no argue here, just do not yet understood why not to optimize rxe and enjoy both worlds? i.e. why to have new sw device just for loopback. We had this discussion in the past, recall Leon? What is the difference now? Yuval
