>> On Sun, 2015-01-25 at 19:09 +0200, Sagi Grimberg wrote:
>> Bound workqueues might be too restrictive since they allow
>> only a single core per session for processing completions.
>> WQ_UNBOUND will allow bouncing to another CPU if the running
>> CPU is currently busy. Luckily, our workqueues are NUMA aware
>> and will first try to bounce within the same NUMA socket.
>> My measurements with NULL backend devices show that there is
>> no (noticeable) additional latency as a result of the change.
>> I'd expect even to gain performance when working with fast
>> devices that also allocate MSIX interrupt vectors.
>>
>> While we're at it, make it WQ_HIGHPRI since processing
>> completions is really a high priority for performance.
>>
>> This one is an RFC since I'd like to ask the users to try out
>> this patch and report the results.
>>
>> Signed-off-by: Sagi Grimberg <sagig@xxxxxxxxxxxx>
>> ---
>> drivers/infiniband/ulp/isert/ib_isert.c | 3 ++-
>> 1 files changed, 2 insertions(+), 1 deletions(-)
>>
>> diff --git a/drivers/infiniband/ulp/isert/ib_isert.c b/drivers/infiniband/ulp/isert/ib_isert.c
>> index dafb3c5..6b5ce34 100644
>> --- a/drivers/infiniband/ulp/isert/ib_isert.c
>> +++ b/drivers/infiniband/ulp/isert/ib_isert.c
>> @@ -3320,7 +3320,8 @@ static int __init isert_init(void)
>> {
>> int ret;
>>
>> - isert_comp_wq = alloc_workqueue("isert_comp_wq", 0, 0);
>> + isert_comp_wq = alloc_workqueue("isert_comp_wq",
>> + WQ_UNBOUND | WQ_HIGHPRI, 0);
>> if (!isert_comp_wq) {
>> isert_err("Unable to allocate isert_comp_wq\n");
>> ret = -ENOMEM;
>
> Moussa has been using the WQ_UNBOUND bit here along with a mlx4 driver
> change to increase the number of EQs available for some time now, with
> impressive small block performance results..
>
I'm specifically interested in performance in NUMA systems (2 or more sockets). I'm wandering if this was the case in Mossa's tests. No arguments that this patch helps performance in a single socket systems.
> I'm going to merge this into target-pending/for-next for now, and can
> drop it later if it ends up being problematic.
> Btw Moussa, what ever happened to the mlx4 driver change..?
>
Mossa's mlx4 patch wasn't the correct way to handle this BUG (mlx4 reserved almost all core EQs for mlx4_en and only 3 for RoCE). The correct fix is heading upstream and hopefully be included in 3.20.
Sagi.--
To unsubscribe from this list: send the line "unsubscribe target-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
