On Wed, Dec 09, 2015 at 02:21:54PM +0200, Moni Shoua wrote:
On Mon, Dec 7, 2015 at 10:42 PM, Dennis Dalessandro
<dennis.dalessandro@xxxxxxxxx> wrote:
The following series implements rdmavt. This is the rdma verbs transport
software library which will help to solve the problem of code duplication
between hardware drivers when it comes to a verbs implementation.
Rdmavt is basically just another verbs provider and lives in the Infiniband tree
in a new sw directory. It provides a software implementation of the Infiniband
verbs API. More details can be found in the following threads:
http://www.spinics.net/lists/linux-rdma/msg29064.html
http://www.spinics.net/lists/linux-rdma/msg29922.html
This patch series is based on 4.4-rc3.
---
Hi Denny
Can you tell how you see the end of this project?
Wouldn't it be right to start with goals and design RFC with more
details than the one you have in the links above, like target drivers
and interfaces?
Our main goal here is to get hfi1 out of staging. *The* blocker for that is
the code duplication. Naturally, the target drivers are qib and hfi1. We
realize it can and should also be used for soft-roce, which is why we are
keeping the development in the open. The design is pretty simple, take
qib/hfi1 common code and move it to a common location with some glue to make
it all work.
As to the interfaces, I don't think we need anything set in stone. In
general you want to avoid changing APIs. However, even if we do have to
change the interface later to accommodate soft-roce or something else we
haven't even thought of yet, that is not a huge deal.
But since you've already posted this, can we at least have this
discussion now before you proceed so we can all have a chance to
influence the final result?
To be more specific, I always imagined the send path like this
[1] Application -> [2] ib_core -> [3] rdmavt -> [4] rvt-backend
The interfaces from [1] to [3] are like the ones we have today and IMO
the one from [3] t0 [4] should look like
int qib_verbs_send(struct qib_qp *qp, struct qib_ib_header *hdr,
u32 hdrwords, struct qib_sge_state *ss, u32 len);
The send path is but one interface. I will try and get a patch out for
comment on that very soon. Although that's probably pretty close. Perhaps
you are concerned about rdmavt doing more than it should? We have very
specific ways of sending packets with our PIO and SDMA engines for instance.
However that sort of detail is hardware specific (is not even the same
between qib and hfi1) and doesn't belong in rdmavt. Thus the interface to
the driver will be a pretty generic one, like what you have shown.
With what I see now, what RDMAVT does is to forward the call to to the
rvt-backend as is. I'm sure that you are not going to keep it this
way or the goal for removing code duplication won't be achieved.
However, I still would like to know about the plans for being there.
Correct. This way we always have a 100% functional driver. For parts which
rdmavt does not yet implement it just passes through, so to speak. Although
technically the call just bypasses rdmavt, it doesn't really "forward"
anything. This lets us do the development incrementally.
Also, you you've out yourself as the maintainer of this project. You
deserve it technically of course but since RDMA_VT is not going to
serve just Intel's backends, wouldn't it be right if we add other
co-maintainers that have an interest on this project?
Who do you have in mind? Adding a co-maintainer is something that I think
would come eventually and am not against it at all. Perhaps once soft-roce
has been added and is supported in rdmavt. Of course this is also all under
the purview of the subsystem maintainer, Doug.
To conclude, RDMA_VT is a project with importance for more than one
vendor and as such it needs to be developed in cooperation or else it
would not serve its purpose.
I think we have all agreed that this is the case. The code has been on
GitHub and we have had a couple of short threads on the mailing list. We
have accepted input from other vendors, and have even included patches
contributed by Kamal in this submission. As has already been seen, posting
to the list has gotten us much more participation in the form of a number of
comments, which is what we want. However our top priority as I said above is
to get the TODO item done for hfi1.
We will _not_ do anything in rdmavt which makes it impossible or overly hard
to add soft-roce support. That is why you are seeing the development as it
progresses. If you see anything which you believe to be contrary to that
please raise the issue. Is there anything in this patchset that should be
called out?
-Denny
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
