> Do you make tests nvme + mlx5 + net ns in your host? Can it work? Sort of, but not really. In our last test, we configured a virtual function and put it in the netns context, but also configured a physical function outside the netns context. TCP NVMe connections always used the correct interface. However, the RoCEv2 NVMe connection always used the physical function, regardless of the user space netns context of the nvme-cli process. When we ran "ip link set <physical function> down" the RoCEv2 NVMe connections stopped working, but TCP NVMe connections were fine. We'll be doing more tests today to make sure we're not doing something wrong. Thanks, Mark On Thu, Apr 13, 2023 at 7:22 AM Zhu Yanjun <yanjun.zhu@xxxxxxxxx> wrote: > > > 在 2023/4/13 5:01, Mark Lehrer 写道: > >> the fabrics device and writing the host NQN etc. Is there an easy way > >> to prove that rdma_resolve_addr is working from userland? > > Actually I meant "is there a way to prove that the kernel > > rdma_resolve_addr() works with netns?" > > I think rdma_resolve_addr can work with netns because rdma on mlx5 can > work well with netns. > > I do not delve into the source code. But IMO, this function should be > used in rdma on mlx5. > > > > > It seems like this is the real problem. If we run commands like nvme > > discover & nvme connect within the netns context, the system will use > > the non-netns IP & RDMA stacks to connect. As an aside - this seems > > like it would be a major security issue for container systems, doesn't > > it? > > Do you make tests nvme + mlx5 + net ns in your host? Can it work? > > Thanks > > Zhu Yanjun > > > > > I'll investigate to see if the fabrics module & nvme-cli have a way to > > set and use the proper netns context. > > > > Thanks, > > Mark
