On Sat, Jun 08, 2019 at 11:32:24AM +0300, Leon Romanovsky wrote: > On Thu, Jun 06, 2019 at 01:05:11PM +0300, Yuval Shaia wrote: > > The virtual address that is registered is used as a base for any address > > passed later in post_recv and post_send operations. > > > > On some virtualized environment this is not correct. > > > > A guest cannot register its memory so hypervisor maps the guest physical > > address to a host virtual address and register it with the HW. Later on, > > at datapath phase, the guest fills the SGEs with addresses from its > > address space. > > Since HW cannot access guest virtual address space an extra translation > > is needed to map those addresses to be based on the host virtual address > > that was registered with the HW. > > This datapath interference affects performances. > > > > To avoid this, a logical separation between the address that is > > registered and the address that is used as a offset at datapath phase is > > needed. > > This separation is already implemented in the lower layer part > > (ibv_cmd_reg_mr) but blocked at the API level. > > > > Fix it by introducing a new API function which accepts an address from > > guest virtual address space as well, to be used as offset for later > > datapath operations. > > > > Signed-off-by: Yuval Shaia <yuval.shaia@xxxxxxxxxx> > > --- > > v0 -> v1: > > * Change reg_mr callback signature instead of adding new callback > > * Add the new API to libibverbs/libibverbs.map.in > > v1 -> v2: > > * Do not modify reg_mr signature for version 1.0 > > * Add note to man page > > v2 -> v3: > > * Rename function to reg_mr_iova (and arg-name to iova) > > * Some checkpatch issues not related to this fix but detected now > > * s/__FUNCTION__/__func > > * WARNING: function definition argument 'void *' should > > also have an identifier name > > v3 -> v4: > > * Fix commit message as suggested by Adit Ranadiv > > * Add support for efa > > --- > > libibverbs/driver.h | 2 +- > > libibverbs/dummy_ops.c | 2 +- > > libibverbs/libibverbs.map.in | 1 + > > libibverbs/man/ibv_reg_mr.3 | 15 +++++++++++++-- > > libibverbs/verbs.c | 23 ++++++++++++++++++++++- > > libibverbs/verbs.h | 7 +++++++ > > providers/bnxt_re/verbs.c | 6 +++--- > > providers/bnxt_re/verbs.h | 2 +- > > providers/cxgb3/iwch.h | 4 ++-- > > providers/cxgb3/verbs.c | 9 +++++---- > > providers/cxgb4/libcxgb4.h | 4 ++-- > > providers/cxgb4/verbs.c | 9 +++++---- > > providers/efa/verbs.c | 4 ++-- > > providers/efa/verbs.h | 2 +- > > providers/hfi1verbs/hfiverbs.h | 4 ++-- > > providers/hfi1verbs/verbs.c | 8 ++++---- > > providers/hns/hns_roce_u.h | 2 +- > > providers/hns/hns_roce_u_verbs.c | 6 +++--- > > providers/i40iw/i40iw_umain.h | 3 ++- > > providers/i40iw/i40iw_uverbs.c | 8 ++++---- > > providers/ipathverbs/ipathverbs.h | 4 ++-- > > providers/ipathverbs/verbs.c | 8 ++++---- > > providers/mlx4/mlx4.h | 4 ++-- > > providers/mlx4/verbs.c | 7 +++---- > > providers/mlx5/mlx5.h | 4 ++-- > > providers/mlx5/verbs.c | 7 +++---- > > providers/mthca/ah.c | 3 ++- > > providers/mthca/mthca.h | 4 ++-- > > providers/mthca/verbs.c | 6 +++--- > > providers/nes/nes_umain.h | 3 ++- > > providers/nes/nes_uverbs.c | 9 ++++----- > > providers/ocrdma/ocrdma_main.h | 4 ++-- > > providers/ocrdma/ocrdma_verbs.c | 10 ++++------ > > providers/qedr/qelr_main.h | 4 ++-- > > providers/qedr/qelr_verbs.c | 11 ++++------- > > providers/qedr/qelr_verbs.h | 4 ++-- > > providers/rxe/rxe.c | 6 +++--- > > providers/vmw_pvrdma/pvrdma.h | 4 ++-- > > providers/vmw_pvrdma/verbs.c | 7 +++---- > > 39 files changed, 133 insertions(+), 97 deletions(-) > > You need to bump PABI in main CmakeList.txt file, otherwise "old > providers" won't work with new libibverbs. The P stands for "Private" (asking so i'll know to look carefully at section "Private symbols in libibverbs"). Just bump 22 to 23? How about PACKAGE_VERSION? nothing there? > > Also I don't see any changes in debian/ folder, which is not right too. debian/changelog? debian/control? debian/libibverbs1.symbols? Anything needs to be done for other distro? redhat, suse? Shall i take commit 75c65bbca as an example? > > Thanks
