Thank you so much for the answers. However, the case I was meaning to discuss is where there are multiple client/threads using different send queues, sending RDMA atomic and write operations concurrently to the same HCA as the responder. I understand that even for single send queue, there could be inconsistent ordering between RDMA atomic operations and reads and that is why we need to use FENCE, but I suppose it does not apply in my case? Thanks, Dong Young > On Aug 25, 2016, at 5:11 PM, Asgeir Eiriksson <asgeir.eiriksson@xxxxxxxxx> wrote: > > Hello Dong, > > The ordering rules for iWARP RDMA are spelled out in RFC7306 (see https://tools.ietf.org/pdf/rfc7306.pdf section 5, pp. 24 for the two ordering cases you asked about) and RFC5040 has the rest of the iWARP RDMA ordering rules > > The answer to your question is the same for iWARP RDMA as that provided by Majd for IB > > Regards, > > ‘Asgeir > > >> On Aug 25, 2016, at 12:11 AM, Majd Dibbiny <majd@xxxxxxxxxxxx> wrote: >> >> >>> On Aug 24, 2016, at 11:43 PM, Dong Young Yoon <dyoon@xxxxxxxxx> wrote: >>> >>> Hi, >>> >>> My name is Dong Young and I am a PhD student at the University of Michigan. >>> Recently, we have been working on a research topic related to RDMA and had an observation where there is a race condition between RDMA atomic operations (e.g., fetch-and-add) and RDMA 1-sided writes. >>> We would like to verity whether what we observed is actually true (i.e., no atomicity guaranteed between RDMA atomic operation and RDMA read/write). >> Hi Dong, >> >> If you post an atomic operation followed by RDMA read/write, you need a fence between the two, because the second one might start processing before the first one has completed processing. >> >> In order to do that, you need to set the IBV_SEND_FENCE in the send_flags of the wr. >> >>> Is it really true? I have also contacted other fellow researchers who worked on RDMA, but they do not seem to be 100% sure on this matter. >>> Thank you so much in advance for your help. >>> >>> Thanks, >>> Dong Young-- >>> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in >>> the body of a message to majordomo@xxxxxxxxxxxxxxx >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in >> the body of a message to majordomo@xxxxxxxxxxxxxxx >> More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
