On 12/5/2018 1:39 PM, Jason Gunthorpe wrote:
On Wed, Dec 05, 2018 at 01:06:50PM -0500, Dennis Dalessandro wrote:
subsystems exist to put common things under. They are not a marketing
term. If a driver doesn't use the subsystem's support code, or
implement the subsystems APIs, why should it be in the subsystem?
Why should we be against the subsystem evolving though? I'm not saying we
allow just anything, just that we leave it open for discussion, *if*
something comes up.
Where is the evolution here though? No new services were added to the
core to support this unique driver.
Which is why I think we treat this as RFC. It's a start, but there is a
ways to go.
Now if we do want to say a driver must support verbs it can't be
wishy-washy. We should specify exactly the set of verbs that must be
supported. The IBTA has the notion of some mandatory verbs right, so should
that be the low bar?
I think the unambiguous low bar is enough implemented to run
NVMEoF. That is definitely 'RDMA common verbs' hardware, IMHO.
Sounds reasonable to me.
Some random proprietary MPI acceleration HW is probably something
else. We aren't trying to squeeze Aries/Gemini/PAMI/etc into the RDMA
verbs model after all.
If you want to evolve the RDMA subsystem into some kind of home for
'dynamic process protected direct IO' - then sure, go do that. Get
Kenneth, Gal, etc together and define a suitable user API for creating
totally opaque QPs and we can put a proper core subsystem API together
for this.
This could be interesting, and this would be evolution.
Defining yet another driver private QP type and ENOTSUPPing half the
subsystem is just a another usnic hack.
Not arguing that. And with the guideline above of being enough to run
NVMEoF Amazon has a bar to shoot for to be compatible with the subsystem.
point where there is enough support for the core to manage the
device from a high level, and bigger picture things like SELinux and
containers, but leave the data path up to the driver/userspace to go
figure out?
So, we just ignored usnic for containers SELinux and other modern
stuff. This is another reason why it is a bad idea.
Regardless I think it's clear the low bar for the subsystem going
forward is verbs support sufficient to run NVMEoF. I don't have any
objections.
Now the question becomes, does rdma-core (user tool) need to be
supported? Or is having a libfabrics provider going to be sufficient? We
have danced around the topic, but let's make it clear.
-Denny
