> On Aug 18, 2017, at 12:52 PM, Doug Ledford <dledford@xxxxxxxxxx> wrote: > > On Wed, 2017-08-16 at 11:50 -0400, Chuck Lever wrote: >>> On Aug 8, 2017, at 2:28 PM, Chuck Lever <chuck.lever@xxxxxxxxxx> >>> wrote: >>> >>> >>>> On Aug 2, 2017, at 3:20 AM, Leon Romanovsky <leon@xxxxxxxxxx> >>>> wrote: >>>> >>>> On Tue, Aug 01, 2017 at 05:06:51PM -0400, Chuck Lever wrote: >>>>> Since I converted NFSD to use the new rdma_rw API, I've been >>>>> struggling with how NFSD determines the size of its send CQ, >>>>> and how >>>>> it provisions its send queue limit (ie, when it waits for more >>>>> send >>>>> queue entries to become available). >>>>> >>>>> The problem arises because rdma_rw hides the exact number of >>>>> send >>>>> WQEs it has provisioned. It determines this number based on the >>>>> MR >>>>> registration mode (FRWR for iWARP devices, local DMA rkey for >>>>> others), but the mode itself is no longer exposed to ULPs. >>>>> >>>>> Thus when FRWR is in use, rdma_rw adds more WQEs, but NFSD is >>>>> no >>>>> longer aware of this, does not provision a larger CQ, and does >>>>> not >>>>> raise its send queue accounting limit. That means the extra >>>>> WQEs are >>>>> never really used. >>>>> >>>>> At LSF this year, Christoph and Sagi proposed a simple API that >>>>> expose information about how many CQ entries are needed so that >>>>> ULPs >>>>> can provision their CQs more accurately. >>>> >>>> Indeed clean API. Does it make sense to get rid of >>>> sc_max_requests and >>>> use this exported value directly? >>> >>> I'm going to consider this clean up for a later patch. >>> >>> So, how shall I submit this new API for upstream? I propose that >>> I submit all three of these via Bruce's tree, with an Acked-by >>> from Doug on the "Add rdma_rw_mr_payload()" patch, if Doug >>> approves. >> >> Doug? > > Your proposal is acceptable. Ack given for 2nd patch to rdma core. Thanks! -- Chuck Lever -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
