On 03-Jan-19 12:27, Leon Romanovsky wrote: > On Thu, Jan 03, 2019 at 11:54:01AM +0200, Gal Pressman wrote: >> On 02-Jan-19 20:12, Jason Gunthorpe wrote: >>> On Sun, Dec 23, 2018 at 05:14:34PM +0200, Gal Pressman wrote: >>> >>>> I'd really like to get to a point where we know how our next >>>> submission is going to look like. I understand that the lack of user >>>> code accompanied to the driver is an issue - that won't be the case >>>> in my next submission. >>> >>> Well, your submission is up to you :) There seems to have been quite a >>> lot missing from the first go around, so I think you'll get more >>> comments. >>> >>> You definitely need some kind of userspace though, we are not merging >>> any kernel uapi stuff without some kind of userspace. >> >> Agreed. >> >>> >>>> As I said, my proposal is to add libibverbs support (with standard >>>> QPT_UD and QP states) and specification for SRD. In addition, I can >>>> come up with an RFC for hiding EFA from in-kernel API as you >>>> suggested, although we will probably revisit this in the future. >>>> Will this be acceptable? >>> >>> I have a feeling EFA will hit the same problems as usnic - libibverbs >>> will be difficult to extend to expose the various special things (how >>> will AH's even work? What about MTU?). >>> >>> Although we are in a better place now, so maybe those problems are now >>> solvable. >>> >>> Unless the libfabric EFA provider will use libibverbs for the kernel >>> interface I'm not sure I see the point in making something for >>> libibverbs that no-one is expected to use?? >>> >>> Jason >>> >> >> The libfabric EFA provider doesn't use libibverbs, but it does use the same >> kernel verbs interface and kABI. >> >> If a libibverbs provider must be accompanied to each driver in the subsystem >> we'll do our best to comply. You know this library better than I (obviously :)), >> so there might be small issues we'll have to face. I guess we'll still have to >> use an EFA libfabric provider that calls libibverbs instead of using the >> existing libfabric verbs provider directly. >> >> If a libfabric provider that uses the same verbs kernel ABI is acceptable, >> that's what we currently have and that's what fits our customers needs the most >> and does not force them to use two different libraries for EFA usage. >> >> Whatever we do, I'll make sure to submit the userspace bits for review as well. > > I'm slightly lost here, are you saying that EFA doesn't use libfabric > verbs interface and implemented ibv_* calls by itself? > > Because libfabric verbs provider uses libibverbs behind the scenes. > > Thanks > Correct, since there's no libibverbs EFA provider, libfabric EFA provider cannot make use of ibv_* calls/verbs provider.
