kindly ping On 15/03/2022 18:18, Li Zhijian wrote: > Hey folks, > > These patches are going to implement a *NEW* RDMA opcode "RDMA FLUSH". > In IB SPEC 1.5[1], 2 new opcodes, ATOMIC WRITE and RDMA FLUSH were > added in the MEMORY PLACEMENT EXTENSIONS section. > > This patchset makes SoftRoCE support new RDMA FLUSH on RC and RD service. > > You can verify the patchset by building and running the rdma_flush example[2]. > server: > $ ./rdma_flush_server -s [server_address] -p [port_number] > client: > $ ./rdma_flush_client -s [server_address] -p [port_number] > > - We introduce new packet format for FLUSH request. > - We introduce FLUSH placement type attributes to HCA > - We introduce FLUSH access flags that users are able to register with > > [1]: https://www.infinibandta.org/wp-content/uploads/2021/08/IBTA-Overview-of-IBTA-Volume-1-Release-1.5-and-MPE-2021-08-17-Secure.pptx > [2]: https://github.com/zhijianli88/rdma-core/tree/rdma-flush > > CC: Xiao Yang <yangx.jy@xxxxxxxxxxx> > CC: y-goto@xxxxxxxxxxx > CC: Jason Gunthorpe <jgg@xxxxxxxx> > CC: Zhu Yanjun <zyjzyj2000@xxxxxxxxx > CC: Leon Romanovsky <leon@xxxxxxxxxx> > CC: Bob Pearson <rpearsonhpe@xxxxxxxxx> > CC: Mark Bloch <mbloch@xxxxxxxxxx> > CC: Wenpeng Liang <liangwenpeng@xxxxxxxxxx> > CC: Aharon Landau <aharonl@xxxxxxxxxx> > CC: Tom Talpey <tom@xxxxxxxxxx> > CC: "Gromadzki, Tomasz" <tomasz.gromadzki@xxxxxxxxx> > CC: Dan Williams <dan.j.williams@xxxxxxxxx> > CC: linux-rdma@xxxxxxxxxxxxxxx > CC: linux-kernel@xxxxxxxxxxxxxxx > > Can also access the kernel source in: > https://github.com/zhijianli88/linux/tree/rdma-flush > Changes log > V3: > - Just rebase and commit log and comment updates > - delete patch-1: "RDMA: mr: Introduce is_pmem", which will be combined into "Allow registering persistent flag for pmem MR only" > - delete patch-7 > > V2: > RDMA: mr: Introduce is_pmem > check 1st byte to avoid crossing page boundary > new scheme to check is_pmem # Dan > > RDMA: Allow registering MR with flush access flags > combine with [03/10] RDMA/rxe: Allow registering FLUSH flags for supported device only to this patch # Jason > split RDMA_FLUSH to 2 capabilities > > RDMA/rxe: Allow registering persistent flag for pmem MR only > update commit message, get rid of confusing ib_check_flush_access_flags() # Tom > > RDMA/rxe: Implement RC RDMA FLUSH service in requester side > extend flush to include length field. # Tom and Tomasz > > RDMA/rxe: Implement flush execution in responder side > adjust start for WHOLE MR level # Tom > don't support DMA mr for flush # Tom > check flush return value > > RDMA/rxe: Enable RDMA FLUSH capability for rxe device > adjust patch's order. move it here from [04/10] > > Li Zhijian (7): > RDMA: Allow registering MR with flush access flags > RDMA/rxe: Allow registering persistent flag for pmem MR only > RDMA/rxe: Implement RC RDMA FLUSH service in requester side > RDMA/rxe: Implement flush execution in responder side > RDMA/rxe: Implement flush completion > RDMA/rxe: Enable RDMA FLUSH capability for rxe device > RDMA/rxe: Add RD FLUSH service support > > drivers/infiniband/core/uverbs_cmd.c | 17 +++ > drivers/infiniband/sw/rxe/rxe_comp.c | 4 +- > drivers/infiniband/sw/rxe/rxe_hdr.h | 48 +++++++++ > drivers/infiniband/sw/rxe/rxe_loc.h | 2 + > drivers/infiniband/sw/rxe/rxe_mr.c | 36 ++++++- > drivers/infiniband/sw/rxe/rxe_opcode.c | 35 ++++++ > drivers/infiniband/sw/rxe/rxe_opcode.h | 3 + > drivers/infiniband/sw/rxe/rxe_param.h | 4 +- > drivers/infiniband/sw/rxe/rxe_req.c | 15 ++- > drivers/infiniband/sw/rxe/rxe_resp.c | 135 ++++++++++++++++++++++-- > include/rdma/ib_pack.h | 3 + > include/rdma/ib_verbs.h | 29 ++++- > include/uapi/rdma/ib_user_ioctl_verbs.h | 2 + > include/uapi/rdma/ib_user_verbs.h | 19 ++++ > include/uapi/rdma/rdma_user_rxe.h | 7 ++ > 15 files changed, 346 insertions(+), 13 deletions(-) >
