On Tue, 2 Apr 2019 11:14:59 +0300, Leon Romanovsky wrote: > On Mon, Apr 01, 2019 at 11:32:21AM -0700, Jakub Kicinski wrote: > > On Mon, 1 Apr 2019 09:28:16 +0300, Leon Romanovsky wrote: > > > On Sun, Mar 31, 2019 at 02:06:33PM -0700, Jakub Kicinski wrote: > > > > On Sun, 31 Mar 2019 19:44:42 +0300, Leon Romanovsky wrote: > > > > > From: Leon Romanovsky <leonro@xxxxxxxxxxxx> > > > > > > > > > > From Ariel, > > > > > > > > > > This series of patches adds user space managed steering infrastructure > > > > > to the mlx5_ib driver. > > > > > > > > > > User space managed steering requires the means to access a dedicated > > > > > memory space that is used by the device to store the packet steering > > > > > and header modification tables and rules in order to manage them directly > > > > > without the device's firmware involvement. This dedicated memory is part > > > > > of the ICM memory space. > > > > > > > > > > The changes are introducing the mlx5_ib API to allocate, deallocate and > > > > > register this dedicated SW ICM memory via the existing device memory API > > > > > using a private attribute which specifies the memory type. > > > > > > > > > > The allocated memory itself is not IO mapped and user can only access it > > > > > using remote RDMA operations. > > > > > > > > > > In addition, the series exposed the ICM address of the receive transport > > > > > interface (TIR) of Raw Packet and RSS QPs to user since they are required > > > > > to properly create and insert steering rules that direct flows to these QPs. > > > > > > > > Hi Leon! Out of curiosity what protocols are you guys steering here? > > > > Is this some form of RoCE or IB? Is there a good resource to read up on > > > > the DEVX stuff? (I presume this series is related?) > > > > > > mlx5 devices steering operates on packets without relation to specific > > > protocol. The decision to steer is based on HW capabilities and user > > > configuration for matching specific fields. Such configuration is done > > > through kernel and through DEVX. > > > > > > General description of DEVX can be found in manuals of libmlx5. > > > https://github.com/linux-rdma/rdma-core/blob/master/providers/mlx5/man/mlx5dv_devx_obj_create.3.md > > > https://github.com/linux-rdma/rdma-core/blob/master/providers/mlx5/man/ > > > > Thanks for those, looks quite powerful. What's your primary use case > > here? Is there some flow steering and encapsulation going on in RDMA > > world? > > Usual use case for that is in RoCE fabric, but as I said it is not > limited to that. For example, flow steering is used in libpcap [1] > and encapsulation is used for efficient VM to VM communication over > different hosts (I assume exactly as in netdev world). > > In regarding to this patch set, the combination of this steering > functionality with DEVX allows us to speedup greatly buildup of rules > by doing it in powerful host CPU instead of internal to NIC CPU. > > [1] > https://github.com/the-tcpdump-group/libpcap/commit/f5f1484a931851d2d3b67d6033aedea8da5acb42 Interesting, thanks for the info!
