On 09/08/2018 14:41, Alexey Kardashevskiy wrote: > > > On 25/07/2018 19:50, Alexey Kardashevskiy wrote: >> I am trying to pass through a 3D controller: >> [0302]: NVIDIA Corporation GV100GL [Tesla V100 SXM2] [10de:1db1] (rev a1) >> >> which has a quite unique feature as coherent memory directly accessible >> from a POWER9 CPU via an NVLink2 transport. >> >> So in addition to passing a PCI device + accompanying NPU devices, >> we will also be passing the host physical address range as it is done >> on the bare metal system. >> >> The memory on the host is presented as: >> >> === >> [aik@yc02goos ~]$ lsprop /proc/device-tree/memory@42000000000 >> ibm,chip-id 000000fe (254) >> device_type "memory" >> compatible "ibm,coherent-device-memory" >> reg 00000420 00000000 00000020 00000000 >> linux,usable-memory >> 00000420 00000000 00000000 00000000 >> phandle 00000726 (1830) >> name "memory" >> ibm,associativity >> 00000004 000000fe 000000fe 000000fe 000000fe >> === >> >> and the host does not touch it as the second 64bit value of >> "linux,usable-memory" - the size - is null. Later on the NVIDIA driver >> trains the NVLink2 and probes this memory and this is how it becomes >> onlined. >> >> In the virtual environment I am planning on doing the same thing, >> however there is a difference in 64bit DMA handling. The powernv >> platform uses a PHB3 bypass mode and that just works but >> the pseries platform uses DDW RTAS API to achieve the same >> result and the problem with this is that we need a huge DMA >> window to start from zero (because this GPU supports less than >> 50bits for DMA address space) and cover not just present memory >> but also this new coherent memory. >> >> >> This is based on sha1 >> d72e90f3 Linus Torvalds "Linux 4.18-rc6". >> >> Please comment. Thanks. > > > Ping? Ping? > > >> >> >> >> Alexey Kardashevskiy (3): >> powerpc/pseries/iommu: Allow dynamic window to start from zero >> powerpc/pseries/iommu: Force default DMA window removal >> powerpc/pseries/iommu: Use memory@ nodes in max RAM address >> calculation >> >> arch/powerpc/platforms/pseries/iommu.c | 77 ++++++++++++++++++++++++++++++---- >> 1 file changed, 70 insertions(+), 7 deletions(-) >> > -- Alexey
