From: Noa Osherovich <noaos@xxxxxxxxxxxx>
There are PCIe root complex that are able to optimize their
performance when incoming data is multiple full cache lines.
Expose the device capability to report whether the device supports
padding the ending of incoming packets to full cache line, such that
the last upstream write generated by the incoming packet will be a
full cache line.
User should consider several factors before activating this feature:
- In case of high CPU memory load (which may cause PCI backpressure in
turn), if a large percent of the writes are partial cache line, this
feature should be checked as an optional solution.
- This feature might reduce performance if most packets are between
one and two cache lines and PCIe throughput has reached its maximum
capacity. E.g. 65B packet from the network port will lead to 128B
write on PCIe, which may cause trafiic on PCIe to reach high
throughput.
Signed-off-by: Noa Osherovich <noaos@xxxxxxxxxxxx>
Reviewed-by: Yishai Hadas <yishaih@xxxxxxxxxxxx>
---
libibverbs/examples/devinfo.c | 5 ++++-
libibverbs/verbs.h | 1 +
2 files changed, 5 insertions(+), 1 deletion(-)
diff --git a/libibverbs/examples/devinfo.c b/libibverbs/examples/devinfo.c
index 169da2e..d02952e 100644
--- a/libibverbs/examples/devinfo.c
+++ b/libibverbs/examples/devinfo.c
@@ -331,10 +331,13 @@ static void print_odp_caps(const struct ibv_odp_caps *caps)
static void print_device_cap_flags_ex(uint64_t device_cap_flags_ex)
{
uint64_t ex_flags = device_cap_flags_ex & 0xffffffff00000000ULL;
- uint64_t unknown_flags = ~(IBV_DEVICE_RAW_SCATTER_FCS);
+ uint64_t unknown_flags = ~(IBV_DEVICE_RAW_SCATTER_FCS |
+ IBV_DEVICE_PCI_WRITE_END_PADDING);
if (ex_flags & IBV_DEVICE_RAW_SCATTER_FCS)
printf("\t\t\t\t\tRAW_SCATTER_FCS\n");
+ if (ex_flags & IBV_DEVICE_PCI_WRITE_END_PADDING)
+ printf("\t\t\t\t\tPCI_WRITE_END_PADDING\n");
if (ex_flags & unknown_flags)
printf("\t\t\t\t\tUnknown flags: 0x%" PRIX64 "\n",
ex_flags & unknown_flags);
diff --git a/libibverbs/verbs.h b/libibverbs/verbs.h
index 3e543cb..025e321 100644
--- a/libibverbs/verbs.h
+++ b/libibverbs/verbs.h
@@ -142,6 +142,7 @@ enum ibv_device_cap_flags {
* enum range is limited to 4 bytes.
*/
#define IBV_DEVICE_RAW_SCATTER_FCS (1ULL << 34)
+#define IBV_DEVICE_PCI_WRITE_END_PADDING (1ULL << 36)
enum ibv_atomic_cap {
IBV_ATOMIC_NONE,
--
1.8.3.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
