> > Let's start by agreeing on what verbs is: an abstract definition of > > the functionality that a NIC supports. > > No, I don't think that at all. Linux common verbs is a concrete > realization of the the IBA and iWarp specifications - with well > defined behaviors from the application layer right down to exactly > which packets are formed on the wire. I was using definition provided by the IB spec, only replacing HCA with NIC. > The entire point of the verbs ecosystem was to enable heterogenous > interoperability - by being extremely well specified. We now have > many, many vendors implementing interoperable verbs to that spec. Nearly every single command carried across the uverbs ABI carries device specific data. It's not generic at all. Even the values carried in common structures depend not only on the device, but it's current state. Is it IB? RoCEv1? RoCEv2? iWarp? What is the current link? IB? Ethernet? It seems like we're trying to apply a requirement that at odds with reality. Uverbs is a common framework for carrying data and commands between user space and a *specific* kernel device driver. It doesn't guarantee interoperability and user space has to know exactly what device it is communicating with. > > If we want to be more specific, does EFA support some subset of the > > commands and data structures defined ib_user_verbs.h in a manner > > that's similar to other devices? Can the uverbs driver manage > > allocated resources (PDs, MRs, QPs, CQs, AHs) the same as other > > devices. Based on patch 11 and responses to my questions, it looks > > that way to me. There are a couple areas where the functionality is > > missing (e.g. QP state transitions), but the submission looks closer > > than not. > > I think you are stretching into your 'abstract verbs' idea to claim > the objects are the same as other devices: No - I’m looking at the enum and structure definitions in ib_user_verbs.h and seeing if they are used in a similar way. I.e. what changes, if any, were needed to the uverbs driver to support the calls? > A CQ without notification is not really a CQ. Hard polling on a CQ is a valid programming model. Regardless, what does it matter if a device responds to a IB_USER_VERBS_CMD_REQ_NOTIFY_CQ with ENOSYS? Why should that be considered unacceptable? > A QP without the verbs state machine is not really a QP. I see no reason to restrict what states a vendor must implement for their QP, or require that the state changes must be done through a kernel call. I'm pretty sure IB and iWarp define different states anyway. > A PD without rkeys is not really a PD. Some sort of hardware protection mechanism to isolate processes is needed when OS bypass is involved. I'm pretty sure we're never going to agree on any of this. - Sean
