From: Christoph Hellwig > Sent: 09 June 2021 13:53 > > On Wed, Jun 09, 2021 at 02:05:03PM +0300, Leon Romanovsky wrote: > > From: Avihai Horon <avihaih@xxxxxxxxxx> > > > > Relaxed Ordering is a capability that can only benefit users that support > > it. All kernel ULPs should support Relaxed Ordering, as they are designed > > to read data only after observing the CQE and use the DMA API correctly. > > > > Hence, implicitly enable Relaxed Ordering by default for kernel ULPs. > > > > Signed-off-by: Avihai Horon <avihaih@xxxxxxxxxx> > > Signed-off-by: Leon Romanovsky <leonro@xxxxxxxxxx> > > --- > > Changelog: > > v2: > > * Dropped IB/core patch and set RO implicitly in mlx5 exactly like in > > eth side of mlx5 driver. > > This looks great in terms of code changes. But can we please also add a > patch to document that PCIe relaxed ordering is fine for kernel ULP usage > somewhere? Several things need to happen to use relaxed ordering: 1) The pcie target has to be able to use RO for buffer writes and non-RO for control writes. 2) The Linux driver has to enable (1). 3) The generic Linux kernel has to 'not mask' RO on all the PCIe bridges and root. 4) The hardware memory system has to 'not break' when passes a RO write. The comments about the DMA API are almost pointless. They'd only be relevant if the driver has looking for the last byte of a buffer to change - and then assuming the rest is valid. This patch looks like a driver patch - so is changing (2) above. I've seen another patch that (I think) is enabling (3). Although some X86 cpu are known to be broken (aka 4). And I still don't know what a ULP is. I actually know a bit about TLP. I found (and fixed) a bug in the Altera/Intel fpga logic where it didn't like receiving two data TLP in response to a single read TLP. We've also (now) got logic in the fpga that traces all received and sent TLP to an internal memory block. So I can see what the cpus we have actually generate. David - Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
