Hi Bart, > -----Original Message----- > From: Bart Van Assche [mailto:Bart.VanAssche@xxxxxxxxxxx] > Sent: Monday, June 12, 2017 11:29 AM > To: Bart Van Assche <Bart.VanAssche@xxxxxxxxxxx>; Parav Pandit > <parav@xxxxxxxxxxxx>; leon@xxxxxxxxxx; dledford@xxxxxxxxxx > Cc: linux-rdma@xxxxxxxxxxxxxxx; Idan Burstein <idanb@xxxxxxxxxxxx> > Subject: Re: [PATCH rdma-next 0/3] Support out of order data placement > > On Mon, 2017-06-12 at 16:19 +0000, Parav Pandit wrote: > > Hi Bart, > > > > > -----Original Message----- > > > From: linux-rdma-owner@xxxxxxxxxxxxxxx [mailto:linux-rdma- > > > owner@xxxxxxxxxxxxxxx] On Behalf Of Bart Van Assche > > > Sent: Monday, June 12, 2017 10:28 AM > > > To: leon@xxxxxxxxxx; dledford@xxxxxxxxxx > > > Cc: linux-rdma@xxxxxxxxxxxxxxx; Idan Burstein <idanb@xxxxxxxxxxxx> > > > Subject: Re: [PATCH rdma-next 0/3] Support out of order data > > > placement > > > > > > On Mon, 2017-06-12 at 09:49 +0300, Leon Romanovsky wrote: > > > > Out of order data placement capability indicates that if HCA > > > > receives out of order RDMA packets, their data placement can be > > > > done at the desired memory destination given in the packet(s). > > > > This is applicable to RDMA read and write operations. > > > > > > Hello Leon and Parav, > > > > > > Since PCIe writes can be executed out of order, shouldn't that be > > > mentioned in Documentation/infiniband/out_of_order.txt? See also the > > > documentation of the Device Control Register and the Enable Relaxed > > > Ordering bit in the PCIe spec. > > > > There is no change in the way PCIe writes are done with respect to > > this per QP bit. Meaning, if this bit is cleared, PCIe subsystem can > > still do out of order writes depending on relaxed ordering flag. > > Hello Parav, > > That's why I asked to mention PCIe write reordering in out_of_order.txt. > Someone who is reading that document could be mislead to assume that if > the HCA does not reorder writes that no reordering will occur. Make sense. I will update the documentation to describe that PCIe out-of-order writes can still happen. > > > > Additionally, since not handling out-of-order RDMA writes correctly > > > is an ULP bug and since there are more ULPs that handle out-of-order > > > writes correctly than ULPs that don't handle out-of-order writes > > > correctly, if a new flag is introduced, shouldn't that be a flag to disable > out-of-order writes? > > > > Not sure I understood correctly. This bit is not a bug fix for ULPs > > who don't handle out-of-order writes. As I described in Documentation, > > "Out of order data placement capability indicates that if HCA receives > > out of order RDMA packets, their data placement can be done at the > > desired memory destination given in the packet(s). This is applicable > > to RDMA read and write operations." This flag indicates that - in above > condition, HCA can do data placement out-of-order. > > Without enabling this flag, when HCA receives out of order packets, it > > would drop them due to PSN sequence error. > > > > So, - to your question - shouldn't that be a flag to disable out-of-order > writes? > > By default, its disabled at RDMA level. > > I don't think that your last two paragraphs mention anything that had not > yet been mentioned in the four e-mails constituting your patch series. I elaborated two points in email in last two paragraphs to answer your question. (a) Without enabling this out-of-order flag, when HCA receives out of order packets, it would drop them due to PSN sequence error. (b) by default out-of-order is disabled at RDMA level. > Additionally, I think my question was clear and unambiguous. So please > reread my question. I reread your question. Let me try to answer again. > shouldn't that be a flag to disable out-of-order writes? No. There is no need for such flag in context of this patchset because by default out-of-order writes are disabled at RDMA level. > > Thanks, > > Bart. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
