On Mon, Sep 21, 2020 at 02:20:53PM -0700, Jakub Kicinski wrote: > I'd wager the only reason you expose the netdevs at all is for link > settings, stats, packet capture and debug. You'd never run TCP traffic > over those links. And you're fighting against using Linux APIs for the > only real traffic that runs on those links - RDMA(ish) traffic. The usual working flow is to use something like TCP to exchange connection information then pivot to RDMA for the actual data flow. This is why a driver like this could get away with such a low performance implementation for a 100G NIC, it is just application boot metadata being exchanged. Sniffing probably won't work as typically the HW will capture the RoCE traffic before reaching Linux - and the Linux driver couldn't handle a 100G flow anyhow. Stats might not work either. As far as the "usual rules" we do require that accelerator devices sharing a netdev are secure in the concept of netdev userspace security. They can access the assigned RoCEv2 UDP port but cannot do things like forge src IP/MAC addresses, violate VLANs, reach outside net namespaces, capature arbitary traffic, etc. This stuff is tricky and generally requires HW support. Someone has to audit all of this and ensure it meets the netdev security requirements too, otherwise it will need CAP_NET_RAW to function. Obviously this requires seeing enough of a userspace implementation to understand how the design approaches verbs 'Address Handles' and so forth. RDMA HW has had errors before and when discovered it was blocked with CAP_NET_RAW until new chip revs came out, this is something I take very seriously. Jason
