> -----Original Message----- > From: linux-rdma-owner@xxxxxxxxxxxxxxx [mailto:linux-rdma- > owner@xxxxxxxxxxxxxxx] On Behalf Of Sagi Grimberg > Sent: Tuesday, October 07, 2014 8:18 PM > To: linux-rdma@xxxxxxxxxxxxxxx > Cc: bvanassche@xxxxxxx; roland@xxxxxxxxxx; eli@xxxxxxxxxxxx; > ogerlitz@xxxxxxxxxxxx; oren@xxxxxxxxxxxx; sean.hefty@xxxxxxxxx > Subject: [PATCH RFC 0/2] Indirect Fast Memory registration support > > This patch set introduces support for registering a scattered memory area in > an indirect manner. > > Current supported fast registration support has a known limitation where the > memory must be page aligned, meaning memory scatters must be in chunks > of page size except the first which may be in some offset from the start of a > page and the last which may end before the page boundary. > > This can make life hard for ULPs which may serve a scattered list without the > above limitations. Two immediate examples are iSER and SRP that have some > extra logic or work-arounds to handle an arbitrary scatter list of buffers > (which is supported in the entire stack above them). > > The proposed API attempts to follow the well-known fast registration > scheme while allowing the ULP to pass an sg vector rather than a page list > (u64 vector). > I expect ULPs to make use of the global DMA key to populate the lkey of the > sg vector. for example for a scatter list sg of 3 elements the indirect ib_sge > vector will look like: > ib_sge[0]: {dma_key, sg[0]->dma_addr, sg[0]->length} Here each sge can be of max 4G like regular SGEs? And can lkey in each SGE be different? > ib_sge[1]: {dma_key, sg[1]->dma_addr, sg[1]->length} > ib_sge[2]: {dma_key, sg[2]->dma_addr, sg[2]->length} > > Following this patch set I'll send out a usage for this feature in iSER. > In the meantime, I have a working example of krping utility practicing indirect > registration feature at: git://flatbed.openfabrics.org/~sgrimberg/krping.git > (branch: indir_registration) > > Sagi Grimberg (2): > IB/core: Introduce Fast Indirect Memory Registration verbs API > IB/mlx5: Implement Fast Indirect Memory Registration Feature > > drivers/infiniband/core/verbs.c | 29 +++++++++ > drivers/infiniband/hw/mlx5/cq.c | 2 + > drivers/infiniband/hw/mlx5/main.c | 4 + > drivers/infiniband/hw/mlx5/mlx5_ib.h | 20 +++++++ > drivers/infiniband/hw/mlx5/mr.c | 70 ++++++++++++++++++++++- > drivers/infiniband/hw/mlx5/qp.c | 104 > ++++++++++++++++++++++++++++++++++ > include/rdma/ib_verbs.h | 55 +++++++++++++++++- > 7 files changed, 280 insertions(+), 4 deletions(-) > > -- > To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the > body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at > http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
