On Thu, Jan 03, 2019 at 11:34:54PM +0000, Hefty, Sean wrote: > > Nobody has presented another criteria that makes EFA and usnic any > > different, so we are back to the start. Was usnic a bad idea or not? > > How do we support 'usnic' style devices without wrecking the rest of > > the stack? > > Let's start by agreeing on what verbs is: an abstract definition of > the functionality that a NIC supports. No, I don't think that at all. Linux common verbs is a concrete realization of the the IBA and iWarp specifications - with well defined behaviors from the application layer right down to exactly which packets are formed on the wire. The entire point of the verbs ecosystem was to enable heterogenous interoperability - by being extremely well specified. We now have many, many vendors implementing interoperable verbs to that spec. So, to me, anything that is not in that multi-vendor interoperable space is really NOT common verbs. Extended verbs, direct verbs, maybe, but certainly not common verbs. I think you are transfering libfabric thinking to Linux verbs.. > From there, define the uverbs objects in terms of generic behavior, > with addressing kept separate. This is what libfabric did :) > I thought the primary requirement for a driver was to have at least > one user. It doesn't matter if the user is another kernel driver or > a Linux-ABI. It is a hard minimum requirement, not the only one.. Drivers still have to fit into the subsystem. Ie you could use the GPU subsystem to expose EFA, that would be totally nutz, but one could warp the APIs to fit :( > If we want to be more specific, does EFA support some subset of the > commands and data structures defined ib_user_verbs.h in a manner > that's similar to other devices? Can the uverbs driver manage > allocated resources (PDs, MRs, QPs, CQs, AHs) the same as other > devices. Based on patch 11 and responses to my questions, it looks > that way to me. There are a couple areas where the functionality is > missing (e.g. QP state transitions), but the submission looks closer > than not. I think you are stretching into your 'abstract verbs' idea to claim the objects are the same as other devices: A CQ without notification is not really a CQ. A QP without the verbs state machine is not really a QP. A PD without rkeys is not really a PD. Maybe this is OK, but it clearly isn't HW that is implementing verbs. It is some pick and choose of a convenient subset of common verbs.. Jason
