On Mon, Apr 29, 2013 at 10:09 PM, Yan Burman <yanb@xxxxxxxxxxxx> wrote: > > I finally got up to 4.1GB/sec bandwidth with RDMA (ipoib-CM bandwidth is also way higher now). > For some reason when I had intel IOMMU enabled, the performance dropped significantly. > I now get up to ~95K IOPS and 4.1GB/sec bandwidth. > Now I will take care of the issue that I am running only at 40Gbit/s instead of 56Gbit/s, but that is another unrelated problem (I suspect I have a cable issue). > > This is still strange, since ib_send_bw with intel iommu enabled did get up to 4.5GB/sec, so why did intel iommu affect only nfs code? > That's very exciting ! The sad part is that IOMMU has to be turned off. I think ib_send_bw uses a single buffer so the DMA mapping search overhead is not an issue. -- Wendy -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
