On 7/8/2040 1:22 AM, Aya Levin wrote:
On 7/6/2020 10:49 PM, David Miller wrote:
From: Aya Levin <ayal@xxxxxxxxxxxx>
Date: Mon, 6 Jul 2020 16:00:59 +0300
Assuming the discussions with Bjorn will conclude in a well-trusted
API that ensures relaxed ordering in enabled, I'd still like a method
to turn off relaxed ordering for performance debugging sake.
Bjorn highlighted the fact that the PCIe sub system can only offer a
query method. Even if theoretically a set API will be provided, this
will not fit a netdev debugging - I wonder if CPU vendors even support
relaxed ordering set/unset...
On the driver's side relaxed ordering is an attribute of the mkey and
should be available for configuration (similar to number of CPU
vs. number of channels).
Based on the above, and binding the driver's default relaxed ordering
to the return value from pcie_relaxed_ordering_enabled(), may I
continue with previous direction of a private-flag to control the
client side (my driver) ?
I don't like this situation at all.
If RO is so dodgy that it potentially needs to be disabled, that is
going to be an issue not just with networking devices but also with
storage and other device types as well.
Will every device type have a custom way to disable RO, thus
inconsistently, in order to accomodate this?
That makes no sense and is a terrible user experience.
That's why the knob belongs generically in PCI or similar.
Hi Bjorn,
Mellanox NIC supports relaxed ordering operation over DMA buffers.
However for debug prepossess we must have a chicken bit to disable
relaxed ordering on a specific system without effecting others in
run-time. In order to meet this requirement, I added a netdev
private-flag to ethtool for set RO API.
Dave raised a concern regarding embedding relaxed ordering set API per
system (networking, storage and others). We need the ability to manage
relaxed ordering in a unify manner. Could you please define a PCI
sub-system solution to meet this requirement?
Aya.
Isn't there a relaxed ordering bit in the PCIe configuration space?
Couldn't you use that as a global indication of if you can support
relaxed ordering or not? Reading through the spec it seems like that is
kind of the point of the config space bit in the Device Control
register. If the bit is not set there then you shouldn't be able to use
relaxed ordering in the device.
Then it is just a matter of using setpci to enable/disable it.
Thanks.
- Alex
