On 25/07/2018 19:50, Alexey Kardashevskiy wrote: > I am trying to pass through a 3D controller: > [0302]: NVIDIA Corporation GV100GL [Tesla V100 SXM2] [10de:1db1] (rev a1) > > which has a quite unique feature as coherent memory directly accessible > from a POWER9 CPU via an NVLink2 transport. > > So in addition to passing a PCI device + accompanying NPU devices, > we will also be passing the host physical address range as it is done > on the bare metal system. > > The memory on the host is presented as: > > === > [aik@yc02goos ~]$ lsprop /proc/device-tree/memory@42000000000 > ibm,chip-id 000000fe (254) > device_type "memory" > compatible "ibm,coherent-device-memory" > reg 00000420 00000000 00000020 00000000 > linux,usable-memory > 00000420 00000000 00000000 00000000 > phandle 00000726 (1830) > name "memory" > ibm,associativity > 00000004 000000fe 000000fe 000000fe 000000fe > === > > and the host does not touch it as the second 64bit value of > "linux,usable-memory" - the size - is null. Later on the NVIDIA driver > trains the NVLink2 and probes this memory and this is how it becomes > onlined. > > In the virtual environment I am planning on doing the same thing, > however there is a difference in 64bit DMA handling. The powernv > platform uses a PHB3 bypass mode and that just works but > the pseries platform uses DDW RTAS API to achieve the same > result and the problem with this is that we need a huge DMA > window to start from zero (because this GPU supports less than > 50bits for DMA address space) and cover not just present memory > but also this new coherent memory. > > > This is based on sha1 > d72e90f3 Linus Torvalds "Linux 4.18-rc6". > > Please comment. Thanks. Ping? > > > > Alexey Kardashevskiy (3): > powerpc/pseries/iommu: Allow dynamic window to start from zero > powerpc/pseries/iommu: Force default DMA window removal > powerpc/pseries/iommu: Use memory@ nodes in max RAM address > calculation > > arch/powerpc/platforms/pseries/iommu.c | 77 ++++++++++++++++++++++++++++++---- > 1 file changed, 70 insertions(+), 7 deletions(-) > -- Alexey -- To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
