On 4/7/2015 8:34 AM, Michael Wang wrote:
/** + * cap_read_multi_sge - Check if the port of device has the capability + * RDMA Read Multiple Scatter-Gather Entries. + * + * @device: Device to be checked + * @port_num: Port number of the device + * + * Return 0 when port of the device don't support + * RDMA Read Multiple Scatter-Gather Entries. + */ +static inline int cap_read_multi_sge(struct ib_device *device, u8 port_num) +{ + return !rdma_transport_iwarp(device, port_num); +}
This just papers over the issue we discussed earlier. How *many* entries does the device support? If a device supports one, or two, is that enough? How does the upper layer know the limit? This needs an explicit device attribute, to be fixed properly.
+ +/** * cap_ipoib - Check if the port of device has the capability * IP over Infiniband. * diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index e011027..604d035 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -118,8 +118,8 @@ static void rdma_build_arg_xdr(struct svc_rqst *rqstp, static int rdma_read_max_sge(struct svcxprt_rdma *xprt, int sge_count) { - if (rdma_node_get_transport(xprt->sc_cm_id->device->node_type) == - RDMA_TRANSPORT_IWARP) + if (!cap_read_multi_sge(xprt->sc_cm_id->device, + xprt->sc_cm_id->port_num)) return 1; else return min_t(int, sge_count, xprt->sc_max_sge);
This is incorrect. The RDMA Read max is not at all the same as the max_sge. It is a different operation, with a different set of work request parameters. In other words, the above same comment applies.
diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c index 4e61880..e75175d 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_transport.c +++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c @@ -979,8 +979,8 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) /* * Determine if a DMA MR is required and if so, what privs are required */ - switch (rdma_node_get_transport(newxprt->sc_cm_id->device->node_type)) { - case RDMA_TRANSPORT_IWARP: + if (rdma_transport_iwarp(newxprt->sc_cm_id->device, + newxprt->sc_cm_id->port_num)) { newxprt->sc_dev_caps |= SVCRDMA_DEVCAP_READ_W_INV;
Do I read this correctly that it is forcing the "read with invalidate" capability to "on" for all iWARP devices? I don't think that is correct, for the legacy devices you're also supporting.
@@ -992,8 +992,8 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) dma_mr_acc = IB_ACCESS_LOCAL_WRITE; } else need_dma_mr = 0; - break; - case RDMA_TRANSPORT_IB: + } else if (rdma_ib_mgmt(newxprt->sc_cm_id->device, + newxprt->sc_cm_id->port_num)) { if (!(newxprt->sc_dev_caps & SVCRDMA_DEVCAP_FAST_REG)) { need_dma_mr = 1; dma_mr_acc = IB_ACCESS_LOCAL_WRITE;
Now I'm even more confused. How is the presence of IB management related to needing a privileged lmr? -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html