On Wed, Sep 25, 2024 at 07:06:41AM +0200, Jiri Slaby wrote: > On 25. 06. 24, 17:31, Vidya Sagar wrote: > > PCIe ACS settings control the level of isolation and the possible P2P > > paths between devices. With greater isolation the kernel will create > > smaller iommu_groups and with less isolation there is more HW that > > can achieve P2P transfers. From a virtualization perspective all > > devices in the same iommu_group must be assigned to the same VM as > > they lack security isolation. > > > > There is no way for the kernel to automatically know the correct > > ACS settings for any given system and workload. Existing command line > > options (ex:- disable_acs_redir) allow only for large scale change, > > disabling all isolation, but this is not sufficient for more complex cases. > > > > Add a kernel command-line option 'config_acs' to directly control all the > > ACS bits for specific devices, which allows the operator to setup the > > right level of isolation to achieve the desired P2P configuration. > > The definition is future proof, when new ACS bits are added to the spec > > the open syntax can be extended. > > > > ACS needs to be setup early in the kernel boot as the ACS settings > > effect how iommu_groups are formed. iommu_group formation is a one > > time event during initial device discovery, changing ACS bits after > > kernel boot can result in an inaccurate view of the iommu_groups > > compared to the current isolation configuration. > > > > ACS applies to PCIe Downstream Ports and multi-function devices. > > The default ACS settings are strict and deny any direct traffic > > between two functions. This results in the smallest iommu_group the > > HW can support. Frequently these values result in slow or > > non-working P2PDMA. > > > > ACS offers a range of security choices controlling how traffic is > > allowed to go directly between two devices. Some popular choices: > > - Full prevention > > - Translated requests can be direct, with various options > > - Asymmetric direct traffic, A can reach B but not the reverse > > - All traffic can be direct > > Along with some other less common ones for special topologies. > > > > The intention is that this option would be used with expert knowledge > > of the HW capability and workload to achieve the desired > > configuration. > > Hi, > > this breaks ACS on some platforms (in 6.11). See: > https://bugzilla.suse.com/show_bug.cgi?id=1229019 #regzbot introduced: 47c8846a ("PCI: Extend ACS configurability") #regzbot link: https://bugzilla.suse.com/show_bug.cgi?id=1229019
