On Tue, Dec 15, 2020 at 05:12:33PM -0800, Edwin Peer wrote: > 1) More than 256 SFs are possible: Maybe it's about time PCI-SIG > addresses this limit for VFs? They can't, the Bus/Device/Function is limited by protocol and changing that would upend the entire PCI world. Instead PCI-SIG said PASID is the way forward. > If that were the only problem with VFs, then fixing it once there > would be cleaner. Maybe, but half the problem with VFs is how HW expensive they are. The mlx5 SF version is not such a good example, but Intel has shown in other recent patches, like for their idxd, that the HW side of an ADI can be very simple and hypervisor emulation can build a simple HW capability into a full ADI for assignment to a guest. A lot of the trappings that PCI-SIG requires to be implemented in HW for a VF, like PCI config space, MSI tables, BAR space, etc. is all just dead weight when scaling up to 1000's of VFs. The ADI scheme is not that bad, the very simplest HW is just a queue that can have all DMA contained by a PASID and can trigger an addr/data interrupt message. Much less HW costly than a SRIOV VF. Regardless, Intel kicked this path off years ago when they published their SIOV cookbook and everyone started integrating PASID support into their IOMMUs and working on ADIs. The mlx5 SFs are kind of early because the HW is flexible enough to avoid the parts of SIOV that are not ready or widely deployed yet, like IMS and PASID. > Like you, I would also prefer a more common infrastructure for > exposing something based on VirtIO/VMDq as the container/VM facing > netdevs. A major point is to get switchdev. > I also don't see how this tackles container/VF portability, > migration of workloads, kernel network stack bypass, or any of the > other legacy limitations regarding SR-IOV VFs It isn't ment too. SF/ADI are just a way to have more VF's than PCI SIG can support.. Jason
