On 3/8/2017 11:56 PM, Jason Gunthorpe wrote:
On Wed, Mar 08, 2017 at 11:27:51PM +0200, Yishai Hadas wrote:
As of that any command that needs the lock must be done before the
flush and delays the hardware from see the BF data immediately.
The counter point is that the unlock macro can combine the WC flushing
barrier with the spinlock atomics, reducing the amount of global
fencing. If you remove the macro your remove that optimization.
The optimization is done as part of mmio_wc_spinlock() for X86, this
macro is still used.
Why not do this:
- mlx4_bf_copy(ctx->bf_page + ctx->bf_offset, (unsigned long *) ctrl,
- align(size * 16, 64));
-
+ tmp_bf_offset = ctx->bf_offset;
ctx->bf_offset ^= ctx->bf_buf_size;
The above 2 commands are still delaying the writing to the NIC comparing
the original code where it was done in one command after mlx4_bf_copy().
+ mlx4_bf_copy(ctx->bf_page + tmp_bf_offset, (unsigned long *) ctrl,
+ align(size * 16, 64));
The candidate mlx4 code will be as follows, similar logic will be in mlx5.
@@ -477,22 +474,18 @@ out:
ctrl->owner_opcode |= htonl((qp->sq.head & 0xffff) << 8);
ctrl->bf_qpn |= qp->doorbell_qpn;
+ ++qp->sq.head;
/*
* Make sure that descriptor is written to memory
* before writing to BlueFlame page.
*/
- mmio_wc_start();
-
- ++qp->sq.head;
-
- pthread_spin_lock(&ctx->bf_lock);
+ mmio_wc_spinlock(&ctx->bf_lock);
mlx4_bf_copy(ctx->bf_page + ctx->bf_offset, (unsigned
long *) ctrl, align(size * 16, 64));
mmio_flush_writes();
ctx->bf_offset ^= ctx->bf_buf_size;
pthread_spin_unlock(&ctx->bf_lock);
} else if (nreq) {
qp->sq.head += nreq;
diff --git a/util/udma_barrier.h b/util/udma_barrier.h
index 9e73148..ec14dd3 100644
--- a/util/udma_barrier.h
+++ b/util/udma_barrier.h
@@ -33,6 +33,8 @@
#ifndef __UTIL_UDMA_BARRIER_H
#define __UTIL_UDMA_BARRIER_H
+#include <pthread.h>
+
/* Barriers for DMA.
These barriers are explicitly only for use with user DMA
operations. If you
@@ -222,4 +224,17 @@
*/
#define mmio_ordered_writes_hack() mmio_flush_writes()
+/* Higher Level primitives */
+
+/* Do mmio_wc_start and grab a spinlock */
+static inline void mmio_wc_spinlock(pthread_spinlock_t *lock)
+{
+ pthread_spin_lock(lock);
+#if !defined(__i386__) && !defined(__x86_64__)
+ /* For x86 the serialization within the spin lock is enough to
+ * strongly order WC and other memory types. */
+ mmio_wc_start();
+#endif
+}
+
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
