On Thu, Nov 28, 2024 at 05:21:42PM +0800, Zelong Yue wrote: > 1. IPv6 addresses must be in the same subnet > 2. The 'local' routing table must have lower priority than our custom policy > routes > 3. When IPv6 addresses are in different subnets, enabling RDMA loopback > breaks TCP loopback > functionality unless packet forwarding is enabled (which isn't feasible > in our DC > environment). We're still investigating a more elegant solution that > wouldn't require > packet forwarding or impact TCP loopback functionality. What do you mean by packet forwarding? For multi-NIC RDMA loopback to work the network must forward packets externally from one NIC to another. Internal-to-the-node loopback is not possible. If that forwarding works for RDMA I would expect it to work for TCP too. For single-NIC, I believe you can use lo to trigger an internal NIC RDMA loopback as well. In all cases TCP and RDMA traffic should be aligned, otherwise it becomes impossible to use normal IP tools to debug the RDMA traffic. > Given that RDMA loopback has different requirements from TCP/UDP > loopback, maybe they should follow distinct routing logic. It cannot, things must be consisent. Jason
