On 10/30/2014 4:36 PM, Bart Van Assche wrote:
On 10/30/14 15:19, Sagi Grimberg wrote:
My impression here that in the default settings, on a 1 NUMA node with
8 cores, 2 different srp connections (using 4 channels each) will be
associated with comp vectors 0-3. while it could potentially use
vectors 4-7 and reduce possible mutual interference. right?
Hello Sagi,
That's correct. For this example if use of all completion vectors is
desired additional configuration is required, e.g. by setting ch_count
to 8 in /etc/modprobe.d/ib_srp.conf.
That is why I think that the user is still expected to be aware of
the configuration in order to get max performance. I would like to
see best performance to "just work". For example, I don't see any sort
of sw queue count to configure, it "just works".
Now I also agree with this may mean more (or sometimes way more)
resources, but I suggest that if we go with default of 4 per numa node
we should take care of such situations.
I'm not strict about this wrt to this patch set. But I think we should
consider this bit.
By the way, I'm not sure it is
possible to avoid manual configuration and tuning entirely. As an
example, with a six core CPU at the initiator side and with
hyperthreading enabled (12 CPU threads in total) I see higher IOPS
results with ch_count=6 compared to ch_count=8 or ch_count=12.
I have
not tried to determine why but maybe this is because ch_count values
that are below the number of CPU threads cause some interrupt coalescing.
I'm not aware of any implicit interrupt coalescing effect...
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
