On 30.04.24 16:52, Chuck Lever III wrote:
On Apr 30, 2024, at 10:45 AM, Zhu Yanjun <zyjzyj2000@xxxxxxxxx> wrote:
On 30.04.24 16:13, Chuck Lever III wrote:
On Apr 30, 2024, at 9:58 AM, Zhu Yanjun <yanjun.zhu@xxxxxxxxx> wrote:
On 30.04.24 15:42, Chuck Lever III wrote:
On Apr 30, 2024, at 3:26 AM, Zhu Yanjun <zyjzyj2000@xxxxxxxxx> wrote:
On 29.04.24 17:25, cel@xxxxxxxxxx wrote:
From: Chuck Lever <chuck.lever@xxxxxxxxxx>
Avoid getting work queue splats in the system journal by moving
client-side RPC/RDMA transport tear-down into a background process.
I've done some testing of this series, now looking for review
comments.
How to make tests with nfs && rdma? Can you provide some steps or tools?
We are building NFS tests into kdevops:
https://github.com/linux-kdevops/kdevops.git
and there is a config option to use soft iWARP instead of TCP.
Thanks a lot. It is interesting. Have you made tests with RXE instead of iWARP?
If yes, does nfs work well with RXE? I am just curious with nfs && RXE.
Normally nfs works with TCP. Now nfs will use RDMA instead of TCP.
The popular RDMA implementation is RoCEv2 which is based on UDP protocol.
So I am curious if NFS can work well with RXE (RoCEv2 emulation driver) or not.
If the user wants to use nfs in his production hosts, it is possible that nfs will work with RoCEv2 (UDP).
Yes, NFS/RDMA works with rxe and even with rxe mixed with
hardware RoCE. Someone else will have to step in and say
whether it works "well" since I don't use rxe, only CX-5
and newer on 100GbE.
Generally we use siw because our testing environment varies
between all systems on a single local network or hypervisor,
all the way up to NFS/RDMA on VPN and WAN. The rxe driver
doesn't support operation over tunnels, currently.
Thanks a lot. "The rxe driver doesn't support operation over tunnels, currently." Do you mean that rxe can not work well with tun/tap device?
No, rxe cannot be configured to use tunnel devices, AFAIK.
It is possible to add rxe as a second option in kdevops,
but siw has worked for our purposes so far, and the NFS
test matrix is already enormous.
Thanks. If rxe can be as a second option in kdevops, I will make tests with kdevops to check rxe work well or not in the future kernel version.
No new tests are necessary. The only thing missing right
now is the ability to set up rxe devices on all the test
systems.
Got it. Thanks.
Zhu Yanjun
Best Regards,
Zhu Yanjun
Best Regards,
Zhu Yanjun
kdevops includes workflows for fstests, Mora's nfstest, the
git regression suite, and ltp, all of which we use regularly
to test the Linux NFS client and server implementations.
I am interested in nfs && rdma.
Thanks,
Zhu Yanjun
Chuck Lever (4):
xprtrdma: Remove temp allocation of rpcrdma_rep objects
xprtrdma: Clean up synopsis of frwr_mr_unmap()
xprtrdma: Delay releasing connection hardware resources
xprtrdma: Move MRs to struct rpcrdma_ep
net/sunrpc/xprtrdma/frwr_ops.c | 13 ++-
net/sunrpc/xprtrdma/rpc_rdma.c | 3 +-
net/sunrpc/xprtrdma/transport.c | 20 +++-
net/sunrpc/xprtrdma/verbs.c | 173 ++++++++++++++++----------------
net/sunrpc/xprtrdma/xprt_rdma.h | 21 ++--
5 files changed, 125 insertions(+), 105 deletions(-)
base-commit: e67572cd2204894179d89bd7b984072f19313b03
--
Chuck Lever
--
Best Regards,
Yanjun.Zhu
--
Chuck Lever
--
Chuck Lever
--
Best Regards,
Yanjun.Zhu
