Hello Sagi, Christoph, Keith, Jason, Doug, Leon and Co
This patchset adds a new verbs API for T10-PI offload and
implementation for iSER initiator (NVMe-oF/RDMA host side was completed
and will be sent on a different patchset).
This set starts with a few preparation commits to the RDMA/core layer.
It continues with introducing a new MR type IB_MR_TYPE_PI. Using this MR
all the needed mappings will be done in the low level drivers and not
be visible to the ULP. Later patches implement the needed functionality
in the mlx5 layer. As suggested by Sagi, in the new API, the mlx5 driver
will allocate a single internal memory region for the UMR operation to
register both PI and data SG lists and it will look like:
data start meta start
| |
-------------------------
|d1|d2|d3|d4|m1|m2|m3|m4|
-------------------------
The sig_mr stride block would be using the same lkey but different
offsets in it (offset 0 and offset d1+d2+d3+d4). The verbs layer will
use a special mr type that will describe everything and will replace
the old API, that enforce using 3 different memory regions (data_mr,
protection_mr, sig_mr) and their local invalidations. This will ease
the code in the ULP and will improve the abstraction of the HW (see
iSER code changes).
The patchset contains also iSER initator patches that using this new API.
For iSER, the code was tested vs. LIO iSER target using Mellanox's
ConnectX-4/ConnectX-5.
This series applies cleanly on top of kernel 5.0-rc5 tag and iser fix
("efa423925cd4 IB/iser: Pass the correct number of entries for dma mapped SGL").
We should aim to push this code during 5.1 merge window.
Next steps are:
- use new API in iSER LIO target and remove the old API (this will
cause changes in RDMA/core layer).
- merge NVMe-oF/RDMA host side after merging this patchset
- Implement metadata support for NVMe-oF/RDMA target side with new API
---------
Changes since v1:
- Add a missing comma at patch 01/17
- Fix coding style at patches 03/17, 05/17 and 09/17
- Fix srp_map_finish_fr function at patch 04/17
- Rebase the code over 5.0-rc5
---------
Israel Rukshin (8):
RDMA/core: Introduce IB_MR_TYPE_PI and ib_alloc_mr_integrity API
RMDA/core: Introduce ib_scatterlist structure
IB/iser: Embed ib_scatterlist into iser_data_buf
IB/srp: Embed ib_scatterlist into srp_map_state struct
IB/iser: Refactor iscsi_iser_check_protection function
IB/iser: Use IB_WR_REG_PI_MR for PI handover
IB/iser: Remove unused sig_attrs argument
IB/isert: Remove unused sig_attrs argument
Max Gurtovoy (9):
RDMA/core: Introduce new header file for signature operations
RDMA/core: Save the MR type in the ib_mr structure
RDMA/core: Introduce ib_map_mr_sg_pi to map data/protection sgl's
RDMA/core: Add signature attrs element for ib_mr structure
RDMA/mlx5: Implement mlx5_ib_map_mr_sg_pi and
mlx5_ib_alloc_mr_integrity
RDMA/mlx5: Add attr for max number page list length for PI operation
RDMA/mlx5: Pass UMR segment flags instead of boolean
RDMA/mlx5: Update set_sig_data_segment attribute for new signature API
RDMA/mlx5: Introduce and implement new IB_WR_REG_PI_MR work request
drivers/infiniband/core/device.c | 2 +
drivers/infiniband/core/rw.c | 6 +-
drivers/infiniband/core/uverbs_cmd.c | 2 +
drivers/infiniband/core/verbs.c | 127 ++++++++++--
drivers/infiniband/hw/bnxt_re/ib_verbs.c | 5 +-
drivers/infiniband/hw/bnxt_re/ib_verbs.h | 3 +-
drivers/infiniband/hw/cxgb3/iwch_provider.c | 5 +-
drivers/infiniband/hw/cxgb4/iw_cxgb4.h | 3 +-
drivers/infiniband/hw/cxgb4/mem.c | 5 +-
drivers/infiniband/hw/hns/hns_roce_device.h | 3 +-
drivers/infiniband/hw/hns/hns_roce_mr.c | 5 +-
drivers/infiniband/hw/i40iw/i40iw_verbs.c | 8 +-
drivers/infiniband/hw/mlx4/mlx4_ib.h | 3 +-
drivers/infiniband/hw/mlx4/mr.c | 5 +-
drivers/infiniband/hw/mlx5/main.c | 4 +
drivers/infiniband/hw/mlx5/mlx5_ib.h | 12 +-
drivers/infiniband/hw/mlx5/mr.c | 192 +++++++++++++++---
drivers/infiniband/hw/mlx5/qp.c | 248 ++++++++++++++++++++----
drivers/infiniband/hw/nes/nes_verbs.c | 5 +-
drivers/infiniband/hw/ocrdma/ocrdma_verbs.c | 5 +-
drivers/infiniband/hw/ocrdma/ocrdma_verbs.h | 3 +-
drivers/infiniband/hw/qedr/verbs.c | 5 +-
drivers/infiniband/hw/qedr/verbs.h | 3 +-
drivers/infiniband/hw/vmw_pvrdma/pvrdma_mr.c | 5 +-
drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.h | 3 +-
drivers/infiniband/sw/rdmavt/mr.c | 10 +-
drivers/infiniband/sw/rdmavt/mr.h | 3 +-
drivers/infiniband/sw/rxe/rxe_verbs.c | 5 +-
drivers/infiniband/ulp/iser/iscsi_iser.c | 9 +-
drivers/infiniband/ulp/iser/iscsi_iser.h | 48 ++---
drivers/infiniband/ulp/iser/iser_initiator.c | 14 +-
drivers/infiniband/ulp/iser/iser_memory.c | 136 +++++--------
drivers/infiniband/ulp/iser/iser_verbs.c | 91 +++------
drivers/infiniband/ulp/isert/ib_isert.c | 11 +-
drivers/infiniband/ulp/srp/ib_srp.c | 46 +++--
drivers/infiniband/ulp/srp/ib_srp.h | 2 +-
drivers/nvme/host/rdma.c | 6 +-
fs/cifs/smbdirect.c | 7 +-
include/linux/mlx5/qp.h | 3 +-
include/rdma/ib_verbs.h | 163 ++++------------
include/rdma/signature.h | 120 ++++++++++++
net/rds/ib_frmr.c | 9 +-
net/smc/smc_ib.c | 9 +-
net/sunrpc/xprtrdma/frwr_ops.c | 6 +-
44 files changed, 872 insertions(+), 493 deletions(-)
create mode 100644 include/rdma/signature.h
--
2.16.3
