> -----Original Message-----
> From: Jason Gunthorpe
> Sent: Thursday, January 11, 2018 6:45 PM
> To: Yuval Shaia <yuval.shaia@xxxxxxxxxx>
> Cc: Alex Margolin <alexma@xxxxxxxxxxxx>; linux-rdma@xxxxxxxxxxxxxxx
> Subject: Re: [RFC rdma-core 2/2] verbs: Introduce non-contiguous memory
> registration
>
> On Thu, Jan 11, 2018 at 02:22:07PM +0200, Yuval Shaia wrote:
> > > +The following code example demonstrates non-contiguous memory
> > > +registration, by combining two contiguous regions, along with the
> WR-based completion semantic:
> > > +.PP
> > > +.nf
> > > +mr1 = ibv_reg_mr(pd, addr1, len1, 0); if (!mr1) {
> > > + fprintf(stderr, "Failed to create MR #1\en");
> > > + return 1;
> > > +}
> > > +
> > > +mr2 = ibv_reg_mr(pd, addr2, len2, 0); if (!mr2) {
> > > + fprintf(stderr, "Failed to create MR #2\en");
> > > + return 1;
> > > +}
> >
> > So, to register non-contiguous 512 random buffers i would have to
> > create
> > 512 MRs?
I think typically if you have a large amount of buffers - it would be located in fairly close proximity, so you'd prefer one MR to cover all of them and the SGEs will only differ in base address.
Are you proposing the function also replaces ibv_reg_mr() if the user passes multiple unregistered regions?
I could see the benefit, but then we'd require additional parameters (i.e. send_flags) and those MRs couldn't be reused (otherwise need to add output pointers for resulting MRs).
The benefit will probably not be latency, though, since IIRC the MR creation can't really be parallelized.
Yuval - are you aware of a scenario implementing a high amount of ibv_reg_mr() calls?
>
> That is a fair point - I wonder if some of these API should have an
> option to accept a pointer directly? Maybe the driver requires a MR but
> we don't need that as an the API?
>
> Particularly the _sg one..
>
> Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
