On 08/02/2021 23:44, Max Gurtovoy wrote:
On 2/5/2021 2:42 AM, Alexey Kardashevskiy wrote:
On 04/02/2021 23:51, Jason Gunthorpe wrote:
On Thu, Feb 04, 2021 at 12:05:22PM +1100, Alexey Kardashevskiy wrote:
It is system firmware (==bios) which puts stuff in the device tree. The
stuff is:
1. emulated pci devices (custom pci bridges), one per nvlink,
emulated by
the firmware, the driver is "ibmnpu" and it is a part on the nvidia
driver;
these are basically config space proxies to the cpu's side of nvlink.
2. interconnect information - which of 6 gpus nvlinks connected to
which
nvlink on the cpu side, and memory ranges.
So what is this vfio_nvlink driver supposed to be bound to?
The "emulated pci devices"?
Yes.
A real GPU function?
Yes.
A real nvswitch function?
What do you mean by this exactly? The cpu side of nvlink is "emulated
pci devices", the gpu side is not in pci space at all, the nvidia
driver manages it via the gpu's mmio or/and cfg space.
Something else?
Nope :)
In this new scheme which you are proposing it should be 2 drivers, I
guess.
I see.
So should it be nvidia_vfio_pci.ko ? and it will do the NVLINK stuff in
case the class code matches and otherwise just work as simple vfio_pci
GPU ?
"nvidia_vfio_pci" would be too generic, sounds like it is for every
nvidia on every platform. powernv_nvidia_vfio_pci.ko may be.
What about the second driver ? should it be called ibmnpu_vfio_pci.ko ?
This will do.
Jason
--
Alexey
