The patch titled
Subject: fs/epoll: drop ovflist branch prediction
has been added to the -mm tree. Its filename is
fs-epoll-drop-ovflist-branch-prediction.patch
This patch should soon appear at
http://ozlabs.org/~akpm/mmots/broken-out/fs-epoll-drop-ovflist-branch-prediction.patch
and later at
http://ozlabs.org/~akpm/mmotm/broken-out/fs-epoll-drop-ovflist-branch-prediction.patch
Before you just go and hit "reply", please:
a) Consider who else should be cc'ed
b) Prefer to cc a suitable mailing list as well
c) Ideally: find the original patch on the mailing list and do a
reply-to-all to that, adding suitable additional cc's
*** Remember to use Documentation/process/submit-checklist.rst when testing your code ***
The -mm tree is included into linux-next and is updated
there every 3-4 working days
------------------------------------------------------
From: Davidlohr Bueso <dave@xxxxxxxxxxxx>
Subject: fs/epoll: drop ovflist branch prediction
The ep->ovflist is a secondary ready-list to temporarily store events that
might occur when doing sproc without holding the ep->wq.lock. This
accounts for every time we check for ready events and also send events
back to userspace; both callbacks, particularly the latter because of
copy_to_user, can account for a non-trivial time.
As such, the unlikely() check to see if the pointer is being used, seems
both misleading and sub-optimal. In fact, we go to an awful lot of
trouble to sync both lists, and populating the ovflist is far from an
uncommon scenario.
For example, profiling a concurrent epoll_wait(2) benchmark, with
CONFIG_PROFILE_ANNOTATED_BRANCHES shows that for a two threads a 33%
incorrect rate was seen; and when incrementally increasing the number of
epoll instances (which is used, for example for multiple queuing load
balancing models), up to a 90% incorrect rate was seen.
Similarly, by deleting the prediction, 3% throughput boost was seen across
incremental threads.
Link: http://lkml.kernel.org/r/20181108051006.18751-4-dave@xxxxxxxxxxxx
Signed-off-by: Davidlohr Bueso <dbueso@xxxxxxx>
Reviewed-by: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
Cc: Al Viro <viro@xxxxxxxxxxxxxxxxxx>
Cc: Jason Baron <jbaron@xxxxxxxxxx>
Signed-off-by: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
---
fs/eventpoll.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/fs/eventpoll.c~fs-epoll-drop-ovflist-branch-prediction
+++ a/fs/eventpoll.c
@@ -1153,7 +1153,7 @@ static int ep_poll_callback(wait_queue_e
* semantics). All the events that happen during that period of time are
* chained in ep->ovflist and requeued later on.
*/
- if (unlikely(ep->ovflist != EP_UNACTIVE_PTR)) {
+ if (ep->ovflist != EP_UNACTIVE_PTR) {
if (epi->next == EP_UNACTIVE_PTR) {
epi->next = ep->ovflist;
ep->ovflist = epi;
_
Patches currently in -mm which might be from dave@xxxxxxxxxxxx are
fs-epoll-remove-max_nests-argument-from-ep_call_nested.patch
fs-epoll-simplify-ep_send_events_proc-ready-list-loop.patch
fs-epoll-drop-ovflist-branch-prediction.patch
fs-epoll-robustify-ep-mtx-held-checks.patch
fs-epoll-reduce-the-scope-of-wq-lock-in-epoll_wait.patch
fs-epoll-reduce-the-scope-of-wq-lock-in-epoll_wait-fix.patch
fs-epoll-avoid-barrier-after-an-epoll_wait2-timeout.patch
fs-epoll-avoid-barrier-after-an-epoll_wait2-timeout-fix.patch
