On Mon, Jul 01, 2019 at 11:15:42AM +0300, Shamir Rabinovitch wrote: > On Thu, Jun 13, 2019 at 02:09:36PM +0300, Yuval Shaia wrote: > > The virtual address that is registered is used as a base for any address > > passed later in post_recv and post_send operations. > > > > On some virtualized environment this is not correct. > > > > A guest cannot register its memory so hypervisor maps the guest physical > > address to a host virtual address and register it with the HW. Later on, > > at datapath phase, the guest fills the SGEs with addresses from its > > address space. > > Since HW cannot access guest virtual address space an extra translation > > is needed to map those addresses to be based on the host virtual address > > that was registered with the HW. > > This datapath interference affects performances. > > > > To avoid this, a logical separation between the address that is > > registered and the address that is used as a offset at datapath phase is > > needed. > > This separation is already implemented in the lower layer part > > (ibv_cmd_reg_mr) but blocked at the API level. > > > > Fix it by introducing a new API function which accepts an address from > > guest virtual address space as well, to be used as offset for later > > datapath operations. > > > > Also update the PABI to v25 > > > > Signed-off-by: Yuval Shaia <yuval.shaia@xxxxxxxxxx> > > Reviewed-by: Leon Romanovsky <leonro@xxxxxxxxxxxx> > > v0 -> v1: > > * Change reg_mr callback signature instead of adding new callback > > * Add the new API to libibverbs/libibverbs.map.in > > v1 -> v2: > > * Do not modify reg_mr signature for version 1.0 > > * Add note to man page > > v2 -> v3: > > * Rename function to reg_mr_iova (and arg-name to iova) > > * Some checkpatch issues not related to this fix but detected now > > * s/__FUNCTION__/__func > > * WARNING: function definition argument 'void *' should > > also have an identifier name > > v3 -> v4: > > * Fix commit message as suggested by Adit Ranadiv > > * Add support for efa > > v4 -> v5: > > * Update PABI > > * Update debian files > > v5 -> v6: > > * Move the new API to section in libibverbs/libibverbs.map.in > > (IBVERBS_1.7) as pointed out by Mark Haywood > > When will this be pulled in to rdma-core master ? > > I can use this API for the shared PD demo app. I was happy with it, but haven't checked every detail yet Jason
