Hi Jason, > -----Original Message----- > From: Jason Gunthorpe [mailto:jgunthorpe@xxxxxxxxxxxxxxxxxxxx] > Sent: Monday, June 12, 2017 11:56 AM > To: Parav Pandit <parav@xxxxxxxxxxxx> > Cc: Bart Van Assche <Bart.VanAssche@xxxxxxxxxxx>; leon@xxxxxxxxxx; > dledford@xxxxxxxxxx; linux-rdma@xxxxxxxxxxxxxxx; Idan Burstein > <idanb@xxxxxxxxxxxx> > Subject: Re: [PATCH rdma-next 0/3] Support out of order data placement > > On Mon, Jun 12, 2017 at 04:53:28PM +0000, Parav Pandit wrote: > > > > > > > From: Jason Gunthorpe [mailto:jgunthorpe@xxxxxxxxxxxxxxxxxxxx] > > > Sent: Monday, June 12, 2017 11:44 AM > > > To: Parav Pandit <parav@xxxxxxxxxxxx> > > > Cc: Bart Van Assche <Bart.VanAssche@xxxxxxxxxxx>; leon@xxxxxxxxxx; > > > dledford@xxxxxxxxxx; linux-rdma@xxxxxxxxxxxxxxx; Idan Burstein > > > <idanb@xxxxxxxxxxxx> > > > Subject: Re: [PATCH rdma-next 0/3] Support out of order data > > > placement > > > > > > On Mon, Jun 12, 2017 at 04:42:27PM +0000, Parav Pandit wrote: > > > > > > > > I would suggest at least using the inverted sense like Bart > > > > > describes in the kernel - every kernel ULP is safe. > > > > > > > I don't see a need to use inverted sense in code. I can surely > > > > make > > > documentation more descriptive as Bart suggested. > > > > > > If this is 'better' then it should be on as much as possible, and I > > > certianly don't want to see kernel ULPs query caps and other > > > pointless things when they already, necessarily, deal with out of order. > > > > Sure. Kernel ULPs and any other user ULPs can skip query caps. > As you know - capable doesn't mean enabled in query_cap. > No, they can't because only mlx5 accepts the new flag. > By default out-of-order processing of RDMA write packets is disabled even on mlx5 for all QPs. When user requests it to enable during modify_qp() as described in Documentation/out_of_order.txt usage section 3, HCA will process it that way. > This is why inverting the flag in the kernel makes much more sense. When you say invert the flag, which flag do you mean? Capability flag or QP attribute? Capability flag indicates - whether out-of-order processing at RDMA level supported or not. QP attribute indicates - to enable such ooo processing for a particular QP. Let's not mix PCIe write ordering with RDMA level out-of-order processing. IB spec clearly says, as I reiterated in the Documentation/out_of_order.txt - that application should not depend on the write data until write_with_imm arrives or subsequent rdma send arrives at responder side. If some MPI or other application have taken shortcut based on some platform, they have to help themselves. This path is not intent to address such issue. > > Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
