On 2/23/22 12:58 AM, Pearson, Robert B wrote:
After investigation, seems the culprit is commit 647bf13ce944 ("RDMA/rxe: Create duplicate mapping tables for FMRs"). The problem is mr_check_range returns -EFAULT after find iova and length are not valid, so connection between two VMs can't be established. Revert the commit manually or apply below temporary change, rxe works again with rnbd/rtrs though I don't think it is the right thing to do. Could experts provide a proper solution? Thanks.This patch fixed failures in blktests and srp which were discussed at length. See e.g. https://lore.kernel.org/linux-rdma/20210907163939.GW1200268@xxxxxxxx/
Thanks for the link, which reminds me the always_invalidate parameter in rtrs_server.
and related messages. The conclusion was that two mappings were required. One owned by the driver and one by the 'hardware', i.e. bottom half in the rxe case, allowing updating a new mapping while the old one is still active and then switching them. If this case has iova and length not valid as indicated is there a problem with the test case?
And after disable always_invalidate (which is enabled by default), rnbd/rtrs over roce works either. So I suppose there might be potential issue for always_invalidate=Y in
rtrs server side since invalidate works for srp IIUC, @Jinpu. Thanks, Guoqing
