On 3/19/2021 3:44 PM, Jason Gunthorpe wrote:
On Fri, Mar 19, 2021 at 03:22:45PM -0400, Dennis Dalessandro wrote:
[Wan, Kaike] I think that you are referring to PSM2, which uses the
OPA hfi1 driver that is specific to the OPA hardware. PSM3 uses
standard verbs drivers and supports standard RoCE.
Uhhh.. "PSM" has always been about the ipath special char device, and
if I recall properly the library was semi-discontinued and merged into
libfabric.
This driver is intended to work with a fork of the PSM2 library. The PSM2
library which is for Omni-Path is now maintained by Cornelis Networks on our
GitHub. PSM3 is something from Intel for Ethernet. I know it's a bit
confusing.
"a bit" huh?
Just a bit. :)
So here you are talking about a libfabric verbs provider that doesn't
use the ipath style char interface but uses verbs and this rv thing so
we call it a libfabric PSM3 provider because thats not confusing to
anyone at all..
A focus is the Intel RDMA Ethernet NICs. As such it cannot use the
hfi1 driver through the special PSM2 interface.
These are the drivers that aren't merged yet, I see. So why are you
sending this now? I'm not interested to look at even more Intel code
when their driver saga is still ongoing for years.
Rather it works with the hfi1 driver through standard verbs
interface.
But nobody would do that right? You'd get better results using the
hif1 native interfaces instead of their slow fake verbs stuff.
I can't imagine why. I'm not sure what you mean by our slow fake verbs
stuff? We support verbs just fine. It's certainly not fake.
hfi1 calls to the kernel for data path operations - that is "fake" in
my book. Verbs was always about avoiding that kernel transition, to
put it back in betrays the spirit. So a kernel call for rv, or the hfi
cdev, or the verbs post-send is really all a wash.
Probably better to argue that in another thread I guess.
I didn't understand your answer, do you see using this with hfi1 or
not?
I don't see how this could ever use hfi1. So no.
It looks a lot copy&pasted from the hfi1 driver, so now we are on our
third copy of this code :(
I haven't had a chance to look beyond the cover letter in depth at how
things have changed. I really hope it's not that bad.
And why did it suddenly become a ULP that somehow shares uverbs
resources?? I'm pretty skeptical that can be made to work correctly..
I see your point.
I was just providing some background/clarification. Bottom line this has
nothing to do with hfi1 or OPA or PSM2.
-Denny
