Hi Bjorn,
On 10/2/2017 12:35 PM, Bjorn Helgaas wrote:
On Fri, Sep 29, 2017 at 05:47:23PM -0700, Ray Jui wrote:
Hi Bjorn and all,
One of our ARM64 based processor (NS2) has our system address
reversed for PCIe outbound transaction in the range of 0x0000_0000
and 0x1fff_ffff. If we use 1:1 mapping between the system address
and PCIe address, we end up having the PCIe address in the same
range.
In this case, we noticed that certain USB-PCI adapters will decline
access to BARs mapped in this address range with "Unsupported
Request". In this case, if we change the PCIe address range to
something else like 0x8000_0000, it would work without an issue.
This sounds like an issue with the USB PCI adapter that cannot allow
BARs to be used with specific PCIe address range. Is my
understanding correct?
I assume you're seeing "Unsupported Request" completions on the PCIe
link? Can you tell with an analyzer that the TLP actually appears on
the link and it's the device that is responding with UR? I'm just
wondering if there's something in the RC that could be
short-circuiting this. But I guess you mention "certain USB-PCI
adapters", which implies that these BAR addresses work for *some*
devices, just not these particular USB-PCI ones.
Yes, UR is observed with a PCIe analyzer. See attached two JPGs for
captures from PCIe analyzer (one uses PCIe address of 0x0000_0000
and the other uses 0x8000_0000).
And yes, this behavior is only seen with certain PCIe-USB adapters.
Most other PCIe endpoint devices (NVMe SSDs, Ethernet dongle, PCIe
switches, and etc) work fine with PICe address of 0x0000_0000.
AFAIK, there's no PCIe reason why the 0x0000_0000 - 0x1fff_ffff
region shouldn't be used for BARs.
Okay, thanks for helping to confirm that.
It's surprising to me that a device would special-case addresses like
that, but maybe it does.
If this is proven to be a shortcoming of some specific PCIe-USB
adapters (and in fact it seems to be the case so far based on all of
the data points). Does it make any sense to you to have a workaround
done by the root complex, i.e., to avoid using 0x0000_0000 outbound
region; instead, use something else, .e.g, 0x1000_0000
