On 05/16/2016 01:54 PM, David Woodhouse wrote: > On Mon, 2016-05-16 at 13:39 +0300, Nikolay Borisov wrote: >> >> I've observed a strange performance pathology with it when running ipoib >> and using a naive iperf test. My setup has multiple machines with a mix >> of qlogic/mellanox cards, connected via an QLogic 12300 switch. All of >> the nodes are running on 4x 10Gbps. When I run a performance test and >> the mellanox card is a server i.e it is receiving data I get very bad >> performance. By this I mean I cannot get more than 4 gigabits per >> second - very low. 'perf top' clearly shows that the culprit is >> intel_map_page which is being called form the receive path >> of the mellanox adapter: >> >> 84.26% 0.04% ksoftirqd/0 [kernel.kallsyms] [k] intel_map_page >> | >> --- intel_map_page >> | >> |--98.38%-- ipoib_cm_alloc_rx_skb > > Are you *sure* it's disabled? Can you be more specific about where the > time is spent? intel_map_page() doesn't really do much except calling > in to __intel_map_single()... which should return fairly much > immediately. Oops, turned out I had intel_iommu=on, though I'm sure some days ago I didn't and performance suffered. I now tried with this option removed and performance is again back to normal. All in all this driver + intel_iommu is a bad combination I guess. Anyway, sorry for the noise and thanks for the prompt reply. Regards, Nikolay -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
