On 9/29/19 3:47 AM, Jackie Liu wrote:
>
>
>> 在 2019年9月29日，07:23，Jens Axboe <axboe@xxxxxxxxx> 写道：
>>
>> Currently any dependent link is executed from a new workqueue context,
>> which means that we'll be doing a context switch per link in the chain.
>> If we are running the completion of the current request from our async
>> workqueue and find that the next request is a link, then run it directly
>> from the workqueue context instead of forcing another switch.
>>
>> This improves the performance of linked SQEs, and reduces the CPU
>> overhead.
>>
>> Signed-off-by: Jens Axboe <axboe@xxxxxxxxx>
>>
>> ---
>>
>> 2-3x speedup doing read-write links, where the read often ends up
>> blocking. Tested with examples/link-cp.c
>>
>> diff --git a/fs/io_uring.c b/fs/io_uring.c
>> index aa8ac557493c..742d95563a54 100644
>> --- a/fs/io_uring.c
>> +++ b/fs/io_uring.c
>> @@ -667,7 +667,7 @@ static void __io_free_req(struct io_kiocb *req)
>> kmem_cache_free(req_cachep, req);
>> }
>>
>> -static void io_req_link_next(struct io_kiocb *req)
>> +struct io_kiocb *io_req_link_next(struct io_kiocb *req)
>> {
>> struct io_kiocb *nxt;
>>
>> @@ -686,9 +686,19 @@ static void io_req_link_next(struct io_kiocb *req)
>> }
>>
>> nxt->flags |= REQ_F_LINK_DONE;
>> + /*
>> + * If we're in async work, we can continue processing this,
>> + * we can continue processing the chain in this context instead
>> + * of having to queue up new async work.
>> + */
>> + if (current_work())
>> + return nxt;
>> INIT_WORK(&nxt->work, io_sq_wq_submit_work);
>> io_queue_async_work(req->ctx, nxt);
>> + nxt = NULL;
>> }
>> +
>> + return nxt;
>> }
>>
>> /*
>> @@ -707,8 +717,10 @@ static void io_fail_links(struct io_kiocb *req)
>> }
>> }
>>
>> -static void io_free_req(struct io_kiocb *req)
>> +static struct io_kiocb *io_free_req(struct io_kiocb *req)
>> {
>> + struct io_kiocb *nxt = NULL;
>> +
>> /*
>> * If LINK is set, we have dependent requests in this chain. If we
>> * didn't fail this request, queue the first one up, moving any other
>> @@ -719,16 +731,30 @@ static void io_free_req(struct io_kiocb *req)
>> if (req->flags & REQ_F_FAIL_LINK)
>> io_fail_links(req);
>> else
>> - io_req_link_next(req);
>> + nxt = io_req_link_next(req);
>> }
>>
>> __io_free_req(req);
>> + return nxt;
>> }
>>
>
> LGTM, Reviewed-by: Jackie Liu <liuyun01@xxxxxxxxxx>
>
> The function io_free_req has been used not only for free req, but also for the task
> of finding the next link entry. I think it is possible to change a name to avoid
> confusion, of course, only personal opinion.
That's a good point. I also changed how we handle the return of that, so there's
no confusion as to a caller getting a nxt request returned and not handling it.
See here:
http://git.kernel.dk/cgit/linux-block/commit/?h=for-5.5/io_uring&id=778fd7a24868b329ff8da2784fd8ced5e35af78c
I'll send out a v2 that's the above, and your naming suggestion.
--
Jens Axboe
