On Mon, Mar 12, 2018 at 08:21:25PM -0500, Rohit Zambre wrote: > On Mon, Mar 12, 2018 at 6:15 PM, Rohit Zambre <rzambre@xxxxxxx> wrote: > > On Mon, Mar 12, 2018 at 2:37 PM, Jason Gunthorpe <jgg@xxxxxxxx> wrote: > >> On Sun, Mar 11, 2018 at 08:07:43PM -0500, Rohit Zambre wrote: > >> > >>> the different bfregs. However, the Mellanox PRM states that doorbells > >>> to the same UAR page must be serialized. > >> > >> This seems like a nonsense statement to me. doorbell rings are > >> indivisible 64 bit writes, there is no concept of serialization of > >> those writes to a PCI-E device. > > > > In the "Sharing UARs" section, the PRM states "No other DoorBell can > > be rung (or even start ringing) in the midst of an on-going write of a > > DoorBell over a given UAR page... the access to a UAR must be > > synchronized unless an atomic write of 64 bits in a single bus > > operation is guaranteed." > > Since this statement is talking about the 64-bit DoorBell writes, I > took a look at where the DoorBell is being written. mlx5_bf_copy calls > the COPY_64B_NT macro which, for x86, writes 64 bytes into the blue > flame buffer using double quadword (128-bits) move-instructions. mlx5_bf_copy is not the 64 bit doorbell write, it is doing something else. 64 bit doorbell writes are executed atomicly at the PCI-E level or globally locked. You should not look at this stuff in MOFED, it doesn't handle the case where 64 bit writes are split, rdma-core does. Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
