On Mon, Jun 12, 2017 at 03:02:59PM -0600, Jason Gunthorpe wrote: > On Mon, Jun 12, 2017 at 03:57:29PM -0500, Steve Wise wrote: > > > > > When transmitter and receiver is enabled to do so, as I described in > > > > overview section of Documentation, it helps > > > > > (a) to avoid retransmission - improves network utilization > > > > > (b) reduces latency due to timers not kicking in. > > > > > > > > Yes those benefits are clear. I see no reason why it shouldn't always > > > > be > > > > done is my point. Application shouldn't have to care and there is no > > > > need to make this an additional flag. > > > > > > The app cares when data from write 2 can be written at the target before data > > > from write 1, especially if the writes target the same memory buffers. (At least I > > > think this is the intent of exposing this to the app.) > > > > > > Note that the provider can always provide stronger ordering than what the app > > > needs. > > > > My understanding is that IB or IW apps should never assume ingress > > write or read response data is _placed_ into local memory in the > > order it was transmitted from the peer. The only guarantee is that > > the _indication_ of the arrived data preserve the sender's ordering. > > However, I'm thinking that there are applications out there that > > spin polling local memory that is the target of a write or read > > response and assume the last bit of that memory will get written > > last... > > That is with respect to the CPU, but IB requires strong ordering > between messages within the same QP, eg if I do > > RDMA WRITE addr=0 data=1 > RDMA WRITE addr=0 data=2 > RDMA WRITE addr=0 data=3 > RDMA READ addr=0 > > I must always get 3, not something else. > > It would be notable if this 'out of order' feature violated that > invariant, but many ULPs would probably still be OK. > > Frankly, Parav's original message doesn't seem to describe at all what > this is about, so maybe we should all wait until v2, and maybe more > people from Mellanox could contribute to sensibly describing it if > they want it in ibverbs. I sent this patch series in US timezone to allow Parav to answer on the questions if they arise, but for other people from Mellanox it was night and it limited number of participants. Thanks > > Jason
Attachment:
signature.asc
Description: PGP signature
