On Wed, May 13, 2020 at 12:44:16PM -0400, Dennis Dalessandro wrote: > On 5/13/2020 3:18 AM, Leon Romanovsky wrote: > > On Tue, May 12, 2020 at 05:53:34PM -0700, Sagi Grimberg wrote: > > > > > > > > > > > > FMR is not supported on most recent RDMA devices (that use fast memory > > > > > > > > > registration mechanism). Also, FMR was recently removed from NFS/RDMA > > > > > > > > > ULP. > > > > > > > > > > > > > > > > > > Signed-off-by: Israel Rukshin <israelr@xxxxxxxxxxxx> > > > > > > > > > Signed-off-by: Max Gurtovoy <maxg@xxxxxxxxxxxx> > > > > > > > > > drivers/infiniband/ulp/iser/iscsi_iser.h | 79 +---------- > > > > > > > > > drivers/infiniband/ulp/iser/iser_initiator.c | 19 ++- > > > > > > > > > drivers/infiniband/ulp/iser/iser_memory.c | 188 ++------------------------- > > > > > > > > > drivers/infiniband/ulp/iser/iser_verbs.c | 126 +++--------------- > > > > > > > > > 4 files changed, 40 insertions(+), 372 deletions(-) > > > > > > > > > > > > > > > > Can we do an extra step and remove FMR from srp too? > > > > > > > > > > > > > > Which HCA's will be affected by removing FMR support? Or in other words, > > > > > > > when did (Mellanox) HCA's start supporting fast memory registration? I'm > > > > > > > asking this because there is a tradition in the Linux kernel not to > > > > > > > remove support for old hardware unless it is pretty sure that nobody is > > > > > > > using that hardware anymore. > > > > > > > > > > > > We haven't entirely been following that in RDMA.. More like when > > > > > > people can't test any more it can go. > > > > > > > > > > > > For FMR the support was dropped in newer HW so AFAIK, nobody tests > > > > > > this and it just stands in the way of making FRWR work properly. > > > > > > > > > > > > Do the ULPs stop working or do they just run slower with some regular > > > > > > MR flow? > > > > > > > > > > I'm not sure. I do not have access to RDMA adapters that do not support > > > > > FRWR. > > > > > > > > > > A possible test is to check on websites for used products whether old > > > > > RDMA adapters are still available. Is the InfiniHost adapter one of the > > > > > adapters that supports FMR? It seems like that adapter is still easy to > > > > > find. > > > > > > > > I don't know - AFAIK nobody does any testing on those cards any > > > > more, and doesn't test the ULPs either. > > > > > > > > I know Leon has pushed to remove the mthca driver in the past. At one > > > > point there was a suggestion that drivers that do not support FRWR > > > > should be dropped, but I don't remember if mthca is the last one or > > > > not. > > > > > > > > There has been a big push to purge useless old stuff, look at the > > > > entire arch removals for instance. The large RDMA drivers fall under > > > > the same logic, IMHO. > > > > > > I think we should remove this support, if there is a user of this > > > somewhere he can safely use iscsi. Let alone that iser uses the fmr > > > pools which leaves rkeys exposed for caching purposes. So I'd much > > > rather remove it than trying to fix it. > > > > Agree, given the fact that no one is even going to try to fix. > > > > We don't see new contributors in this community who are not affiliated > > with RDMA companies and they are not testing and have no plans to test FMR. > > > > I feel that we can't even say if FMR and old cards work :). > > qib still works. mlx4 works too :), I had in mind more older cards than qib, which was added to the kernel only ten years ago. For example mlx4 was added 13 years ago an mthca before git era (>15 years). Thanks > > -Denny >
