> -----Original Message----- > From: Yuval Shaia <yuval.shaia@xxxxxxxxxx> > Sent: Monday, March 4, 2019 1:56 AM > To: Parav Pandit <parav@xxxxxxxxxxxx> > Cc: Ira Weiny <ira.weiny@xxxxxxxxx>; Leon Romanovsky <leon@xxxxxxxxxx>; > Dennis Dalessandro <dennis.dalessandro@xxxxxxxxx>; bvanassche@xxxxxxx; > linux-rdma@xxxxxxxxxxxxxxx > Subject: Re: [EXPERIMENTAL v1 0/4] RDMA loopback device > > On Fri, Mar 01, 2019 at 06:27:34AM +0000, Parav Pandit wrote: > > > > > What is the real use case for this? > > Use case is fairly simple. Group of developers and users are running VMs > on laptop and in cloud without hw HCAs for devel purposes and automated > unit and Jenkins tests of their application. > > Such tests run for few hundreds of QPs and intent to exchange million > messages. > > This is mainly RoCE users. > > rxe is not fitting these needs with its current state. > > To run RDMA device in a VM even when no real HCA is installed on host i > can suggest QEMU's pvrdma device which is on its way to become a real > product. > Can you please share the github repo or other open source link which provides the host side of emulation code? I am interested to know how does it work on Linux host without modifying kernel, if it is close to production line. Linux host side code RoCE code doesn't support pvrdma's need. It requires ABI changes... different discussion. And after doing all of it, such VM still requires enhanced host. This approach doesn't have any of those limitations. > > > > Bart was looking to run on loopback (similar to other user request I > received) here [1]. > > > > So I am looking at something insanely simple but extendible who can find > more uses as we go forward. > > Suggestion: To enhance 'loopback' performances, can you consider using > shared memory or any other IPC instead of going thought the network stack? > Loopback driver in this patchset doesn't use network stack. It is just 2000 lines of wrapper to memcpy() to enables applications to use rdma. > > null_blk driver started simple, but stability lead to more features, which > grew for 3/4 module parameters to a configfs based options now. > > > > I am considering if rdma subsystem can offer such stable 'lo netdev' or 'null > block device' style rdma device (starting with RoCE, followed by IB link), > would it be more useful. > > Apart from user devel and appl test needs, loopback driver is useful to > develop regression tests for rdma-subsystem. > > > > Chuck or Bart's ULP experience on how they like it would be interesting to > listen to. > > I used rxe for developing nvme fabrics dissector and it was good enough > for few thousands of packets a year back. > > There has been several fixes by well-wishers. > > > > Bart's problem can be partly solvable, using patch [2] + adding lo device > using new netlink rxe command from Steve, didn't try it yet. > > > > [1] https://marc.info/?l=linux-rdma&m=155122131404449&w=2 > > [2] https://patchwork.kernel.org/patch/10831261/
