Am 08.05.2018 um 18:27 schrieb Logan Gunthorpe:
On 08/05/18 01:17 AM, Christian König wrote:
AMD APUs mandatory need the ACS flag set for the GPU integrated in the
CPU when IOMMU is enabled or otherwise you will break SVM.
Well, given that the current set only disables ACS bits on bridges
(previous versions were only on switches) this shouldn't be an issue for
integrated devices. We do not disable ACS flags globally.
Ok, that is at least a step in the right direction. But I think we
seriously need to test that for side effects.
And what exactly is the problem here? I'm currently testing P2P with
GPUs in different IOMMU domains and at least with AMD IOMMUs that works
perfectly fine.
In addition to Stephen's comments, seeing we've established a general
need to avoid the root complex (until we have a whitelist at least) we
must have ACS disabled along the path between the devices. Otherwise,
all TLPs will go through the root complex and if there is no support it
will fail.
Well I'm not an expert on this, but if I'm not completely mistaken that
is not correct.
E.g. transactions are initially send to the root complex for
translation, that's for sure. But at least for AMD GPUs the root complex
answers with the translated address which is then cached in the device.
So further transactions for the same address range then go directly to
the destination.
What you don't want is device isolation, cause in this case the root
complex handles the transaction themselves. IIRC there where also
something like "force_isolation" and "nobypass" parameters for the IOMMU
to control that behavior.
It's already late here, but going to dig up the documentation for that
tomorrow and/or contact a hardware engineer involved in the ACS spec.
Regards,
Christian.
If the consensus is we want a command line option, then so be it. But
we'll have to deny pretty much all P2P transactions unless the user
correctly disables ACS along the path using the command line option and
this is really annoying for users of this functionality to understand
how to do that correctly.
Logan
