Hi Denny, Sean, As Sean explained, relaxed ordering/out-of-order packets, allows making efficient use of network resources. When transmitter and receiver is enabled to do so, as I described in overview section of Documentation, it helps (a) to avoid retransmission - improves network utilization (b) reduces latency due to timers not kicking in. More comments to Sean's question below. > -----Original Message----- > From: Hefty, Sean [mailto:sean.hefty@xxxxxxxxx] > Sent: Monday, June 12, 2017 2:19 PM > To: Dalessandro, Dennis <dennis.dalessandro@xxxxxxxxx>; Parav Pandit > <parav@xxxxxxxxxxxx>; Steve Wise <swise@xxxxxxxxxxxxxxxxxxxxx>; > 'Leon Romanovsky' <leon@xxxxxxxxxx>; 'Doug Ledford' > <dledford@xxxxxxxxxx> > Cc: linux-rdma@xxxxxxxxxxxxxxx; Idan Burstein <idanb@xxxxxxxxxxxx> > Subject: RE: [PATCH rdma-next 0/3] Support out of order data placement > > > I don't understand why the application has to care about this at all. > > If > > the HW wants to place things out of order. Ok go for it. Applications > > should be depending on the correct mechanism to know when it's OK to > > look at the data. Why do they care what order it arrived in? > > > > The only reason to care is if the applications wants to do something > > that is not compliant and look at parts of the data early. Perhaps you > > can explain *how* an application may benefit from out-of-order > > placement and likewise, why it wouldn't want to benefit. > > Relaxed ordering can improve performance by allowing packets to take > multiple paths from the source to destination. FWIW, the ofiwg actually > discussed ordering at length, and libfabric exposes several ordering related > attributes. > > The application usually needs to know what sort of ordering the network is > providing for correct operation. Yes. This flag defines it how RW data can done in relaxed order manner. > For example, can sends be re-ordered by > the network, such that they may be received in the opposite order that they > were sent? No. Only rdma reads and writes. Table 79 of IB spec still holds true. In internal patch review I had the table, but since there was no change, I just removed it. I think it's worth to mention in the out_of_order.txt that it still holds true. I will add this line. I would also document which compliant statements are not applicable when such flag is set. How about having responder side table, similar to table 79 in Documentation/out_of_order.txt That makes it more readable for users. > Can rdma writes be processed out of order, such that the target > data ends up with some sort of interleaving of the written data? Yes. > Are there size limits to ordering constraints? No. > > It's hard to capture this level of detail with a single flag. There's ordering of > data within a single message, as well as ordering of data between messages. Good documentation should describe what a flag does. Flag name itself has RW prefix in it. Are you suggesting there should be two flags - one for within message, one for across message? If so, shouldn't we add those flag when such different implementation arise. I cannot define something that cannot be implemented or tested at this point. This is across the messages. > I don't know how this proposal works with explicit pathing that IB defines, > or path failover. It doesn't change with path failover or explicit path to my knowledge, but I will confirm if there is any change in v1. > > - Sean ��.n��������+%������w��{.n�����{���fk��ܨ}���Ơz�j:+v�����w����ޙ��&�)ߡ�a����z�ޗ���ݢj��w�f
