> On Jul 10, 2017, at 3:13 PM, Sagi Grimberg <sagi@xxxxxxxxxxx> wrote: > > >>>> I thought the FMR page list limit was always 64. I seem to >>>> vaguely recall testing this way back when I split out the >>>> xprtrdma code for handling different registration modes, >>>> and above 64, FMR fell over (at least with CX2 and CX3). >>> >>> Not sure about CX2, but iser runs fmrs with 128 entries perfectly fine >>> on CX3. IIRC CX2 does not support FRWRs, >> CX2 does indeed support both FMR and FRWR. > > Thanks for the info. > >>> CX4 and up does not support FMRs (which I'd love to kill altogether >>> anyways). >> "Killing in-kernel FMR" might be a good topic for the LPC RDMA >> micro-conference. > > I won't be able to attend, but I'd love if others can discuss it. > I'd like for that to happen, but I'd also like to support as much > non-deprecated devices as possible. In fact, until the FMR bug report > came along, I had a "remove fmr" patch for iser in the pipe and held > it back as someone out there is using iser over qib. > > So as much as I'd like that to happen, so we can just ignore it, I'm > convinced we can do it just yet. >> 1. I concur that it would be good to have some API for advertising >> the limit for FMR. If it's higher than 64 for some devices, that >> will be good for performance. > > That is what I'm trying to achieve. Thumbs up for that. >> 2. But I'm not sure it's always the same as the device's FRWR >> maximum page list depth. If it is always or often smaller, then >> that unfairly limits FRWR. > > The opposite argument can be made too. > > is a separate max_fmr_page_list_len a better way to go? I would prefer that these be exposed as separate per-device limits, yes, since I'll bet they are going to be significantly different values in some cases. >> 3. I'm not at all objecting, but if FMR is to go away, why make >> this change now? > > As I said, I don't know if FMRs are going away, its not on me to > decide. I just tried to help ULPs to not guess the limit. Fair enough. -- Chuck Lever -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
