On Jul 10, 2015, at 10:11 AM, Devesh Sharma <devesh.sharma@xxxxxxxxxxxxx> wrote: > On Fri, Jul 10, 2015 at 6:28 PM, Tom Talpey <tom@xxxxxxxxxx> wrote: >> On 7/10/2015 7:29 AM, Devesh Sharma wrote: >>> >>> we need to honor the max limits of device by checking >>> dev_attr.max_sge? a vendor may not support 4 sges. >> >> >> iWARP requires a minimum of 4 send SGEs (draft-hilland-verbs 8.1.3.2) >> >> An RI MUST support at least four Scatter/Gather Elements per >> Scatter/Gather List when the Scatter/Gather List refers to the Data >> Source of a Send Operation Type or the Data Sink of a Receive >> Operation. An RI is NOT REQUIRED to support more than one >> Scatter/Gather Element per Scatter/Gather List when the >> Scatter/Gather List refers to the Data Source of an RDMA Write. >> >> I'm not certain if IB and RoCE state a similar minimum requirement, >> but it seems a very bad idea to have fewer. > > To my knowledge IBTA Spec do not pose any such minimum requirement. > RoCE also do not puts any minimum requirement. I think its fine if > xprtrdma honors the device limits, thus covering iWARP devices because > all iWARP devices would support minimum 4. > > Chuck would correct me if xprtrdma do have any minimum requirements At least 2 SGEs are required for normal operation. The code in rpcrdma_marshal_req() sets up an iovec for the RPC/RDMA header and one for the RPC message itself. These are used in rpcrdma_ep_post() to SEND each request. Four SGEs are needed only for RDMA_MSGP type requests, but so far I have never seen one of these used. I wonder if that logic can be removed. It is certainly possible to examine the device’s max_sge field in rpcrdma_ep_create() and fail transport creation if the device’s max_sge is less than RPC_MAX_IOVS. -- Chuck Lever -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
