The patch titled
Subject: epoll: make sure all elements in ready list are in FIFO order
has been added to the -mm tree. Its filename is
epoll-make-sure-all-elements-in-ready-list-are-in-fifo-order.patch
This patch should soon appear at
http://ozlabs.org/~akpm/mmots/broken-out/epoll-make-sure-all-elements-in-ready-list-are-in-fifo-order.patch
and later at
http://ozlabs.org/~akpm/mmotm/broken-out/epoll-make-sure-all-elements-in-ready-list-are-in-fifo-order.patch
Before you just go and hit "reply", please:
a) Consider who else should be cc'ed
b) Prefer to cc a suitable mailing list as well
c) Ideally: find the original patch on the mailing list and do a
reply-to-all to that, adding suitable additional cc's
*** Remember to use Documentation/process/submit-checklist.rst when testing your code ***
The -mm tree is included into linux-next and is updated
there every 3-4 working days
------------------------------------------------------
From: Roman Penyaev <rpenyaev@xxxxxxx>
Subject: epoll: make sure all elements in ready list are in FIFO order
Patch series "use rwlock in order to reduce ep_poll_callback()
contention", v3.
The last patch targets the contention problem in ep_poll_callback(), which
can be very well reproduced by generating events (write to pipe or eventfd)
from many threads, while consumer thread does polling.
The following are some microbenchmark results based on the test [1] which
starts threads which generate N events each. The test ends when all events
are successfully fetched by the poller thread:
spinlock
========
threads events/ms run-time ms
8 6402 12495
16 7045 22709
32 7395 43268
rwlock + xchg
=============
threads events/ms run-time ms
8 10038 7969
16 12178 13138
32 13223 24199
According to the results bandwidth of delivered events is significantly
increased, thus execution time is reduced.
This patch (of 4):
All coming events are stored in FIFO order and this is also should be
applicable to ->ovflist, which originally is stack, i.e. LIFO.
Thus to keep correct FIFO order ->ovflist should reversed by adding
elements to the head of the read list but not to the tail.
Link: http://lkml.kernel.org/r/20190103150104.17128-2-rpenyaev@xxxxxxx
Signed-off-by: Roman Penyaev <rpenyaev@xxxxxxx>
Reviewed-by: Davidlohr Bueso <dbueso@xxxxxxx>
Cc: Jason Baron <jbaron@xxxxxxxxxx>
Cc: Al Viro <viro@xxxxxxxxxxxxxxxxxx>
Cc: "Paul E. McKenney" <paulmck@xxxxxxxxxxxxxxxxxx>
Signed-off-by: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
---
fs/eventpoll.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
--- a/fs/eventpoll.c~epoll-make-sure-all-elements-in-ready-list-are-in-fifo-order
+++ a/fs/eventpoll.c
@@ -722,7 +722,11 @@ static __poll_t ep_scan_ready_list(struc
* contain them, and the list_splice() below takes care of them.
*/
if (!ep_is_linked(epi)) {
- list_add_tail(&epi->rdllink, &ep->rdllist);
+ /*
+ * ->ovflist is LIFO, so we have to reverse it in order
+ * to keep in FIFO.
+ */
+ list_add(&epi->rdllink, &ep->rdllist);
ep_pm_stay_awake(epi);
}
}
_
Patches currently in -mm which might be from rpenyaev@xxxxxxx are
epoll-make-sure-all-elements-in-ready-list-are-in-fifo-order.patch
epoll-loosen-irq-safety-in-ep_poll_callback.patch
epoll-unify-awaking-of-wakeup-source-on-ep_poll_callback-path.patch
epoll-use-rwlock-in-order-to-reduce-ep_poll_callback-contention.patch
