On Thu, Dec 23, 2021 at 08:59:14PM +0800, Cheng Xu wrote: > > > On 12/23/21 6:23 PM, Leon Romanovsky wrote: > > On Wed, Dec 22, 2021 at 11:35:44AM +0800, Cheng Xu wrote: > > > > > > > <...> > > > > > > > > > > > > For the ECS instance with RDMA enabled, there are two kinds of devices > > > > > allocated, one for ERDMA, and one for the original netdev (virtio-net). > > > > > They are different PCI deivces. ERDMA driver can get the information about > > > > > which netdev attached to in its PCIe barspace (by MAC address matching). > > > > > > > > This is very questionable. The netdev part should be kept in the > > > > drivers/ethernet/... part of the kernel. > > > > > > > > Thanks > > > > > > The net device used in Alibaba ECS instance is virtio-net device, driven > > > by virtio-pci/virtio-net drivers. ERDMA device does not need its own net > > > device, and will be attached to an existed virtio-net device. The > > > relationship between ibdev and netdev in erdma is similar to siw/rxe. > > > > siw/rxe binds through RDMA_NLDEV_CMD_NEWLINK netlink command and not > > through MAC's matching. > > > > Thanks > > Both siw/rxe/erdma don't need to implement netdev part, this is what I > wanted to express when I said 'similar'. > What you mentioned (the bind mechanism) is one major difference between > erdma and siw/rxe. For siw/rxe, user can attach ibdev to every netdev if > he/she wants, but it is not true for erdma. When user buys the erdma > service, he/she must specify which ENI (elastic network interface) to be > binded, it means that the attached erdma device can only be binded to > the specific netdev. Due to the uniqueness of MAC address in our ECS > instance, we use the MAC address as the identification, then the driver > knows which netdev should be binded to. Nothing prohibits from you to implement this MAC check in RDMA_NLDEV_CMD_NEWLINK. I personally don't like the idea that bind logic is performed "magically". BTW, 1. No module parameters 2. No driver versions Thanks > > Thanks, > Cheng Xu
