Hi Casey:
On 2017/7/8 10:04, Casey Leedom wrote:
> Okay, thanks for the note Alexander. I'll have to look more closely at
> the patch on Monday and try it out on one of the targeted systems to verify
> the semantics you describe.
>
All the modification is only clearing the device's Device Control{Relaxed Ordering
Enable]bit when distinguish that the platform should not support RO and did nothing
to the RC configuration, so I don't think it will break anything compare to the
first version from yours.
> However, that said, there is no way to tell a priori where a device will
> send TLPs. To simply assume that all TLPs will be directed towards the Root
> Complex is a big assumption. Only the device and the code controlling it
> know where the TLPs will be directed. That's why there are changes required
> in the cxgb4 driver. For instance, the code in
> drivers/net/ethernet/chelsio./cxgb4/sge.c: t4_sge_alloc_rxq() knows that
> it's allocating Free List Buffers in Host Memory and that the RX Queues that
> it's allocating in the Hardware will eventually send Ingress Data to those
> Free List Buffers. (And similarly for the Free List Buffer Pointer Queue
> with respect to DMA Reads from the host.) In that routine we explicitly
> configure the Hardware to use/not-use the Relaxed Ordering Attribute via the
> FW_IQ_CMD_FL0FETCHRO and FW_IQ_CMD_FL0DATARO flags. Basically we're
> conditionally setting them based on the desirability of sending Relaxed
> Ordering TLPs to the Root Complex. (And we would perform the same kind of
> check for an nVME application ... which brings us to ...)
>
> And what would be the code using these patch APIs to set up a Peer-to-Peer
> nVME-style application? In that case we'd need the Chelsio adapter's PCIe
> Capability Device Control[Relaxed Ordering Enable] set for the nVME
> application ... and we would avoid programming the Chelsio Hardware to use
> Relaxed Ordering for TLPs directed at the Root Complex. Thus we would be in
> a position where some TLPs being emitted by the device to Peer devices would
> have Relaxed Ordering set and some directed at the Root Complex would not.
> And the only way for that to work is if the source device's Device
> Control[Relaxed Ordering Enable] is set ...
>
> Finally, setting aside my disagreements with the patch, we still have the
> code in the cxgb4 driver which explicitly turns on its own Device
> Control[Relaxed Ordering Enable] in cxgb4_main.c:
> enable_pcie_relaxed_ordering(). So the patch is something of a loop if all
> we're doing is testing our own Relaxed Ordering Enable state ...
>
> Casey
>
> .
>
