On Thu, Dec 06, 2018 at 01:37:07AM +0000, Hefty, Sean wrote: > > > I think the mistake in the tree is either having different protocols > > > map to the same QP type value, or not having a separate protocol > > field > > > provided as part of create QP. > > > > The QP type value specifies the API contract to the verbs consumer. > > I disagree. The primary API contract is determined by the device > attributes and capabilities. The QP type isn't known or used until > well after that. > > The QP type should only determine the behavior of QP related calls: > create/modify/query qp, post_send, post_recv, and ib_wc data. And > even with these, we risk making assumptions. For example, QP type > is independent of memory registration, which affects the input to > send/recv. This is what I ment, the QP type defines the API contract for QP related parts (ie QPT_UD has the 40 byte thing, etc) We only have some fairly small variations due to iWarp, but for the most part this should be enforced. > An issue we've had is linking attributes with orthogonal > APIs/behaviors. Some of these were fixed by adding the > rdma_protocol_xxx() and rdma_cap_xxx() calls. We shouldn't have QP > type carrying broad API implications. IMO, selecting the wire > protocol seems ideal, so I still feel that introducing a new QP type > is sensible in this case. Add documentation for which QP/AH > attributes are valid for each QP type if that helps. QP type doesn't select wire protocol, it fundamentally selects how the send and recv queue works. We have the various cap and related calls in-kernel primarily to deal with the AH differences, with a few related to the MR problem on iWarp. > QPs could be classified at a higher level to define a *general* API > flow that's independent of the protocol. E.g. provide helper > functions to mark a QP as either reliable/unreliable and > connected/unconnected. That would provide a minimal list of what > verbs should be supported, though details on which fields are used > still end up device/protocol specific. This is all implied by the QPT today. You can't have a QPT_RC that isn't reliable, or in-order, or doesn't support RDMA. That would be some other QPT. You can't do UD without the 40 bytes and the usual other things. UD can't start doing RDMA, etc. Jason
