On Mon, Mar 04, 2019 at 02:47:43PM +0000, Parav Pandit wrote: > > > > -----Original Message----- > > From: Yuval Shaia <yuval.shaia@xxxxxxxxxx> > > Sent: Monday, March 4, 2019 1:56 AM > > To: Parav Pandit <parav@xxxxxxxxxxxx> > > Cc: Ira Weiny <ira.weiny@xxxxxxxxx>; Leon Romanovsky <leon@xxxxxxxxxx>; > > Dennis Dalessandro <dennis.dalessandro@xxxxxxxxx>; bvanassche@xxxxxxx; > > linux-rdma@xxxxxxxxxxxxxxx > > Subject: Re: [EXPERIMENTAL v1 0/4] RDMA loopback device > > > > On Fri, Mar 01, 2019 at 06:27:34AM +0000, Parav Pandit wrote: > > > > > > > What is the real use case for this? > > > Use case is fairly simple. Group of developers and users are running VMs > > on laptop and in cloud without hw HCAs for devel purposes and automated > > unit and Jenkins tests of their application. > > > Such tests run for few hundreds of QPs and intent to exchange million > > messages. > > > This is mainly RoCE users. > > > rxe is not fitting these needs with its current state. > > > > To run RDMA device in a VM even when no real HCA is installed on host i > > can suggest QEMU's pvrdma device which is on its way to become a real > > product. > > > Can you please share the github repo or other open source link which provides the host side of emulation code? Official QEMU repo is here: https://github.com/qemu/qemu > I am interested to know how does it work on Linux host without modifying kernel, if it is close to production line. On a host with no real HCA the backend device is RXE. > Linux host side code RoCE code doesn't support pvrdma's need. It requires ABI changes... different discussion. No, no ABI changes is needed, pvrdma is an ibverb client. > > And after doing all of it, such VM still requires enhanced host. This approach doesn't have any of those limitations. Can you elaborate on that? why enhanced host? > > > > > > > Bart was looking to run on loopback (similar to other user request I > > received) here [1]. > > > > > > So I am looking at something insanely simple but extendible who can find > > more uses as we go forward. > > > > Suggestion: To enhance 'loopback' performances, can you consider using > > shared memory or any other IPC instead of going thought the network stack? > > > Loopback driver in this patchset doesn't use network stack. > It is just 2000 lines of wrapper to memcpy() to enables applications to use rdma. gr8!! I had plans to patch RXE to support it but didn't found the time. So actually why not do it in RXE? > > > > null_blk driver started simple, but stability lead to more features, which > > grew for 3/4 module parameters to a configfs based options now. > > > > > > I am considering if rdma subsystem can offer such stable 'lo netdev' or 'null > > block device' style rdma device (starting with RoCE, followed by IB link), > > would it be more useful. > > > Apart from user devel and appl test needs, loopback driver is useful to > > develop regression tests for rdma-subsystem. > > > > > > Chuck or Bart's ULP experience on how they like it would be interesting to > > listen to. > > > I used rxe for developing nvme fabrics dissector and it was good enough > > for few thousands of packets a year back. > > > There has been several fixes by well-wishers. > > > > > > Bart's problem can be partly solvable, using patch [2] + adding lo device > > using new netlink rxe command from Steve, didn't try it yet. > > > > > > [1] https://marc.info/?l=linux-rdma&m=155122131404449&w=2 > > > [2] https://patchwork.kernel.org/patch/10831261/
