I see. Well, those sysfs files are not usable, and I don't think it would be important even if they were usable, but for completeness:
The ioctl returns:
pcie_gen = 1
pcie_num_lanes = 16
pcie_num_lanes = 16
Theoretical bandwidth from those values: 4.0 GB/s
My DMA test shows this write bandwidth: 3.5 GB/s
It matches the expectation.
Let's see the devices (there is only 1 GPU Navi21 in the system):
$ lspci |egrep '(PCI|VGA).*Navi'
0a:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Upstream Port of PCI Express Switch (rev c3)
0b:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Downstream Port of PCI Express Switch
0c:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21 [Radeon RX 6800/6800 XT / 6900 XT] (rev c3)
0a:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Upstream Port of PCI Express Switch (rev c3)
0b:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Downstream Port of PCI Express Switch
0c:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21 [Radeon RX 6800/6800 XT / 6900 XT] (rev c3)
Let's read sysfs:
16
$ cat /sys/bus/pci/devices/0000:0b:00.0/current_link_width
16
$ cat /sys/bus/pci/devices/0000:0c:00.0/current_link_width
16
$ cat /sys/bus/pci/devices/0000:0a:00.0/current_link_speed
2.5 GT/s PCIe
$ cat /sys/bus/pci/devices/0000:0b:00.0/current_link_speed16.0 GT/s PCIe
$ cat /sys/bus/pci/devices/0000:0c:00.0/current_link_speed
16.0 GT/s PCIe
Problem 1: None of the speed numbers match 4 GB/s.
Problem 2: Userspace doesn't know the bus index of the bridges, and it's not clear which bridge should be used.
Problem 3: The PCIe gen number is missing.
That's all irrelevant because all information should be queryable via the INFO ioctl. It doesn't matter what sysfs contains because UMDs shouldn't have to open and parse extra files just to read a couple of integers.
Marek
On Tue, Jan 3, 2023 at 3:31 AM Christian König <ckoenig.leichtzumerken@xxxxxxxxx> wrote:
Sure they can, those files are accessible to everyone.
The massive advantage is that this is standard for all PCIe devices, so it should work vendor independent.
Christian.
Am 02.01.23 um 18:55 schrieb Marek Olšák:
Userspace drivers can't access sysfs.
Marek
On Mon, Jan 2, 2023, 10:54 Christian König <ckoenig.leichtzumerken@xxxxxxxxx> wrote:
That stuff is already available as current_link_speed and current_link_width in sysfs.
I'm a bit reluctant duplicating this information in the IOCTL interface.
Christian.
Am 30.12.22 um 23:07 schrieb Marek Olšák:
For computing PCIe bandwidth in userspace and troubleshooting PCIe
bandwidth issues.
For example, my Navi21 has been limited to PCIe gen 1 and this is
the first time I noticed it after 2 years.
Note that this intentionally fills a hole and padding
in drm_amdgpu_info_device.
Signed-off-by: Marek Olšák <marek.olsak@xxxxxxx>
The patch is attached.
Marek
