On Wed, 2017-08-16 at 11:50 -0400, Chuck Lever wrote:
> > On Aug 8, 2017, at 2:28 PM, Chuck Lever <chuck.lever@xxxxxxxxxx>
> > wrote:
> >
> >
> > > On Aug 2, 2017, at 3:20 AM, Leon Romanovsky <leon@xxxxxxxxxx>
> > > wrote:
> > >
> > > On Tue, Aug 01, 2017 at 05:06:51PM -0400, Chuck Lever wrote:
> > > > Since I converted NFSD to use the new rdma_rw API, I've been
> > > > struggling with how NFSD determines the size of its send CQ,
> > > > and how
> > > > it provisions its send queue limit (ie, when it waits for more
> > > > send
> > > > queue entries to become available).
> > > >
> > > > The problem arises because rdma_rw hides the exact number of
> > > > send
> > > > WQEs it has provisioned. It determines this number based on the
> > > > MR
> > > > registration mode (FRWR for iWARP devices, local DMA rkey for
> > > > others), but the mode itself is no longer exposed to ULPs.
> > > >
> > > > Thus when FRWR is in use, rdma_rw adds more WQEs, but NFSD is
> > > > no
> > > > longer aware of this, does not provision a larger CQ, and does
> > > > not
> > > > raise its send queue accounting limit. That means the extra
> > > > WQEs are
> > > > never really used.
> > > >
> > > > At LSF this year, Christoph and Sagi proposed a simple API that
> > > > expose information about how many CQ entries are needed so that
> > > > ULPs
> > > > can provision their CQs more accurately.
> > >
> > > Indeed clean API. Does it make sense to get rid of
> > > sc_max_requests and
> > > use this exported value directly?
> >
> > I'm going to consider this clean up for a later patch.
> >
> > So, how shall I submit this new API for upstream? I propose that
> > I submit all three of these via Bruce's tree, with an Acked-by
> > from Doug on the "Add rdma_rw_mr_payload()" patch, if Doug
> > approves.
>
> Doug?
Your proposal is acceptable. Ack given for 2nd patch to rdma core.
--
Doug Ledford <dledford@xxxxxxxxxx>
GPG KeyID: B826A3330E572FDD
Key fingerprint = AE6B 1BDA 122B 23B4 265B 1274 B826 A333 0E57 2FDD
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
