On Fri, Sep 9, 2022 12:08 PM Li, Zhijian wrote: > > Daisuke > > Great job. > > I love this feature, before starting reviewing you patches, i tested it with QEMU(with fsdax memory-backend) migration > over RDMA where it worked for MLX5 before. Thank you for the try! Anybody interested in the feature can get the RFC tree from my github. https://github.com/daimatsuda/linux/tree/odp_rfc > > This time, with you ODP patches, it works on RXE though ibv_advise_mr may be not yet ready. ibv_advise_mr(3) is what I referred to as prefetch feature. Currently, libibverbs always returns EOPNOTSUPP error. We need to implement the feature in rxe driver and add a handler to use it from librxe in the future. Thanks, Daisuke > > > Thanks > Zhijian > > > On 07/09/2022 10:42, Daisuke Matsuda wrote: > > Hi everyone, > > > > This patch series implements the On-Demand Paging feature on SoftRoCE(rxe) > > driver, which has been available only in mlx5 driver[1] so far. > > > > [Overview] > > When applications register a memory region(MR), RDMA drivers normally pin > > pages in the MR so that physical addresses are never changed during RDMA > > communication. This requires the MR to fit in physical memory and > > inevitably leads to memory pressure. On the other hand, On-Demand Paging > > (ODP) allows applications to register MRs without pinning pages. They are > > paged-in when the driver requires and paged-out when the OS reclaims. As a > > result, it is possible to register a large MR that does not fit in physical > > memory without taking up so much physical memory. > > > > [Why to add this feature?] > > We, Fujitsu, have contributed to RDMA with a view to using it with > > persistent memory. Persistent memory can host a filesystem that allows > > applications to read/write files directly without involving page cache. > > This is called FS-DAX(filesystem direct access) mode. There is a problem > > that data on DAX-enabled filesystem cannot be duplicated with software RAID > > or other hardware methods. Data replication with RDMA, which features > > high-speed connections, is the best solution for the problem. > > > > However, there is a known issue that hinders using RDMA with FS-DAX. When > > RDMA operations to a file and update of the file metadata are processed > > concurrently on the same node, illegal memory accesses can be executed, > > disregarding the updated metadata. This is because RDMA operations do not > > go through page cache but access data directly. There was an effort[2] to > > solve this problem, but it was rejected in the end. Though there is no > > general solution available, it is possible to work around the problem using > > the ODP feature that has been available only in mlx5. ODP enables drivers > > to update metadata before processing RDMA operations. > > > > We have enhanced the rxe to expedite the usage of persistent memory. Our > > contribution to rxe includes RDMA Atomic write[3] and RDMA Flush[4]. With > > them being merged to rxe along with ODP, an environment will be ready for > > developers to create and test software for RDMA with FS-DAX. There is a > > library(librpma)[5] being developed for this purpose. This environment > > can be used by anybody without any special hardware but an ordinary > > computer with a normal NIC though it is inferior to hardware > > implementations in terms of performance. > > > > [Design considerations] > > ODP has been available only in mlx5, but functions and data structures > > that can be used commonly are provided in ib_uverbs(infiniband/core). The > > interface is heavily dependent on HMM infrastructure[6], and this patchset > > use them as much as possible. While mlx5 has both Explicit and Implicit ODP > > features along with prefetch feature, this patchset implements the Explicit > > ODP feature only. > > > > As an important change, it is necessary to convert triple tasklets > > (requester, responder and completer) to workqueues because they must be > > able to sleep in order to trigger page fault before accessing MRs. I did a > > test shown in the 2nd patch and found that the change makes the latency > > higher while improving the bandwidth. Though it may be possible to create a > > new independent workqueue for page fault execution, it is a not very > > sensible solution since the tasklets have to busy-wait its completion in > > that case. > > > > If responder and completer sleep, it becomes more likely that packet drop > > occurs because of overflow in receiver queue. There are multiple queues > > involved, but, as SoftRoCE uses UDP, the most important one would be the > > UDP buffers. The size can be configured in net.core.rmem_default and > > net.core.rmem_max sysconfig parameters. Users should change these values in > > case of packet drop, but page fault would be typically not so long as to > > cause the problem. > > > > [How does ODP work?] > > "struct ib_umem_odp" is used to manage pages. It is created for each > > ODP-enabled MR on its registration. This struct holds a pair of arrays > > (dma_list/pfn_list) that serve as a driver page table. DMA addresses and > > PFNs are stored in the driver page table. They are updated on page-in and > > page-out, both of which use the common interface in ib_uverbs. > > > > Page-in can occur when requester, responder or completer access an MR in > > order to process RDMA operations. If they find that the pages being > > accessed are not present on physical memory or requisite permissions are > > not set on the pages, they provoke page fault to make pages present with > > proper permissions and at the same time update the driver page table. After > > confirming the presence of the pages, they execute memory access such as > > read, write or atomic operations. > > > > Page-out is triggered by page reclaim or filesystem events (e.g. metadata > > update of a file that is being used as an MR). When creating an ODP-enabled > > MR, the driver registers an MMU notifier callback. When the kernel issues a > > page invalidation notification, the callback is provoked to unmap DMA > > addresses and update the driver page table. After that, the kernel releases > > the pages. > > > > [Supported operations] > > All operations are supported on RC connection. Atomic write[3] and Flush[4] > > operations, which are still under discussion, are also going to be > > supported after their patches are merged. On UD connection, Send, Recv, > > SRQ-Recv are supported. Because other operations are not supported on mlx5, > > I take after the decision right now. > > > > [How to test ODP?] > > There are only a few resources available for testing. pyverbs testcases in > > rdma-core and perftest[7] are recommendable ones. Note that you may have to > > build perftest from upstream since older versions do not handle ODP > > capabilities correctly. > > > > [Future work] > > My next work will be the prefetch feature. It allows applications to > > trigger page fault using ibv_advise_mr(3) to optimize performance. Some > > existing software like librpma use this feature. Additionally, I think we > > can also add the implicit ODP feature in the future. > > > > [1] [RFC 00/20] On demand paging > > https://www.spinics.net/lists/linux-rdma/msg18906.html > > > > [2] [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) > > https://lore.kernel.org/nvdimm/20190809225833.6657-1-ira.weiny@xxxxxxxxx/ > > > > [3] [RESEND PATCH v5 0/2] RDMA/rxe: Add RDMA Atomic Write operation > > https://www.spinics.net/lists/linux-rdma/msg111428.html > > > > [4] [PATCH v4 0/6] RDMA/rxe: Add RDMA FLUSH operation > > https://www.spinics.net/lists/kernel/msg4462045.html > > > > [5] librpma: Remote Persistent Memory Access Library > > https://github.com/pmem/rpma > > > > [6] Heterogeneous Memory Management (HMM) > > https://www.kernel.org/doc/html/latest/mm/hmm.html > > > > [7] linux-rdma/perftest: Infiniband Verbs Performance Tests > > https://github.com/linux-rdma/perftest > > > > Daisuke Matsuda (7): > > IB/mlx5: Change ib_umem_odp_map_dma_single_page() to retain umem_mutex > > RDMA/rxe: Convert the triple tasklets to workqueues > > RDMA/rxe: Cleanup code for responder Atomic operations > > RDMA/rxe: Add page invalidation support > > RDMA/rxe: Allow registering MRs for On-Demand Paging > > RDMA/rxe: Add support for Send/Recv/Write/Read operations with ODP > > RDMA/rxe: Add support for the traditional Atomic operations with ODP > > > > drivers/infiniband/core/umem_odp.c | 6 +- > > drivers/infiniband/hw/mlx5/odp.c | 4 +- > > drivers/infiniband/sw/rxe/Makefile | 5 +- > > drivers/infiniband/sw/rxe/rxe.c | 18 ++ > > drivers/infiniband/sw/rxe/rxe_comp.c | 42 +++- > > drivers/infiniband/sw/rxe/rxe_loc.h | 11 +- > > drivers/infiniband/sw/rxe/rxe_mr.c | 7 +- > > drivers/infiniband/sw/rxe/rxe_net.c | 4 +- > > drivers/infiniband/sw/rxe/rxe_odp.c | 329 ++++++++++++++++++++++++++ > > drivers/infiniband/sw/rxe/rxe_param.h | 2 +- > > drivers/infiniband/sw/rxe/rxe_qp.c | 68 +++--- > > drivers/infiniband/sw/rxe/rxe_recv.c | 2 +- > > drivers/infiniband/sw/rxe/rxe_req.c | 14 +- > > drivers/infiniband/sw/rxe/rxe_resp.c | 175 +++++++------- > > drivers/infiniband/sw/rxe/rxe_resp.h | 44 ++++ > > drivers/infiniband/sw/rxe/rxe_task.c | 152 ------------ > > drivers/infiniband/sw/rxe/rxe_task.h | 69 ------ > > drivers/infiniband/sw/rxe/rxe_verbs.c | 16 +- > > drivers/infiniband/sw/rxe/rxe_verbs.h | 10 +- > > drivers/infiniband/sw/rxe/rxe_wq.c | 161 +++++++++++++ > > drivers/infiniband/sw/rxe/rxe_wq.h | 71 ++++++ > > 21 files changed, 824 insertions(+), 386 deletions(-) > > create mode 100644 drivers/infiniband/sw/rxe/rxe_odp.c > > create mode 100644 drivers/infiniband/sw/rxe/rxe_resp.h > > delete mode 100644 drivers/infiniband/sw/rxe/rxe_task.c > > delete mode 100644 drivers/infiniband/sw/rxe/rxe_task.h > > create mode 100644 drivers/infiniband/sw/rxe/rxe_wq.c > > create mode 100644 drivers/infiniband/sw/rxe/rxe_wq.h > >
