Hi Bjorn, > Subject: Re: [PATCH v2 1/5] PCI/P2PDMA: Don't enforce ACS check for > functions of same device > > On Thu, Oct 24, 2024 at 05:58:48AM +0000, Kasireddy, Vivek wrote: > > > Subject: Re: [PATCH v2 1/5] PCI/P2PDMA: Don't enforce ACS check for > > > functions of same device > > > > > > On Sun, Oct 20, 2024 at 10:21:29PM -0700, Vivek Kasireddy wrote: > > > > Functions of the same PCI device (such as a PF and a VF) share the > > > > same bus and have a common root port and typically, the PF provisions > > > > resources for the VF. Therefore, they can be considered compatible > > > > as far as P2P access is considered. > > I don't understand the "therefore they can be considered compatible" > conclusion. The spec quote below seems like it addresses exactly this > situation: it says ACS can control peer-to-peer requests between VFs. I am only referring to the specific case where the PF is trying to access (P2P) a VF's resource that the PF itself has provisioned. Shouldn't this be considered a valid access? > > > ... > > > I'm not sure what you refer to by "PF provisions resources for the > > > VF". Isn't it *always* the case that the architected PCI > > > resources (BARs) are configured by the PF? It sounds like you're > > > referring to something Intel GPU-specific beyond that? > > > > What I meant to say is that since PF provisions the resources for > > the VF in a typical scenario, > > Are you talking about BARs? As far as I know, the PF BAR assignments > always (not just in typical scenarios) determine the VF BAR > assignments. Right, I am indeed talking about BARs. > > Or are you referring to some other non-BAR resources? > > > they should be automatically P2PDMA compatible particularly when the > > provider is the VF and PF is the client. However, since this cannot > > be guaranteed on all the PCI devices out there for various reasons, > > my objective is to start including the ones that can be tested and > > are known to be compatible (Intel GPUs). > > Regardless of BAR or other VF resources, I don't think VFs are > automatically P2PDMA compatible. I agree that VFs in general are not automatically P2PDMA compatible but a PF and a VF should be considered compatible particularly when the provider is a VF and PF is the client. > For example, PCIe r6.0, sec 6.12.1.2 says: > > For ACS requirements, single-Function devices that are SR-IOV > capable must be handled as if they were Multi-Function Devices. > > ... > > - ACS P2P Request Redirect: must be implemented by Functions that > support peer-to-peer traffic with other Functions. This includes > SR-IOV Virtual Functions (VFs). ACS P2P Request Redirect is > subject to interaction with the ACS P2P Egress Control and ACS > Direct Translated P2P mechanisms (if implemented). Refer to > Section 6.12.3 for more information. When ACS P2P Request > Redirect is enabled in a Multi-Function Device that is not an > RCiEP, peer-to-peer Requests (between Functions of the device) > must be redirected Upstream towards the RC. > > Or do you mean something else by "P2PDMA compatible"? I am no longer making any generic claims about devices' P2PDMA compatibility. Instead, as mentioned above, I am only focused on the interactions between a PF (client) and a VF (provider), particularly with Intel GPUs. More specifically, I am trying to address a use-case where the VF needs to share a buffer with the PF but is unsuccessful because pci_p2pdma_distance_many( provider, client, 1, true) fails (due to ACS redirect being set) although the buffer is located within a BAR resource that the PF has provisioned and has full access to it. Shouldn't this be allowed? Thanks, Vivek > > Bjorn
