On 2019-05-31 11:56, Peter Zijlstra wrote:
On Thu, May 16, 2019 at 10:58:04AM +0200, Roman Penyaev wrote:
Each ep_poll_callback() is called when fd calls wakeup() on epfd.
So account new event in user ring.
The tricky part here is EPOLLONESHOT. Since we are lockless we
have to be deal with ep_poll_callbacks() called in paralle, thus
use cmpxchg to clear public event bits and filter out concurrent
call from another cpu.
Signed-off-by: Roman Penyaev <rpenyaev@xxxxxxx>
Cc: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
Cc: Al Viro <viro@xxxxxxxxxxxxxxxxxx>
Cc: Linus Torvalds <torvalds@xxxxxxxxxxxxxxxxxxxx>
Cc: linux-fsdevel@xxxxxxxxxxxxxxx
Cc: linux-kernel@xxxxxxxxxxxxxxx
diff --git a/fs/eventpoll.c b/fs/eventpoll.c
index 2f551c005640..55612da9651e 100644
--- a/fs/eventpoll.c
+++ b/fs/eventpoll.c
@@ -1407,6 +1407,29 @@ struct file *get_epoll_tfile_raw_ptr(struct
file *file, int tfd,
}
#endif /* CONFIG_CHECKPOINT_RESTORE */
+/**
+ * Atomically clear public event bits and return %true if the old
value has
+ * public event bits set.
+ */
+static inline bool ep_clear_public_event_bits(struct epitem *epi)
+{
+ __poll_t old, flags;
+
+ /*
+ * Here we race with ourselves and with ep_modify(), which can
+ * change the event bits. In order not to override events updated
+ * by ep_modify() we have to do cmpxchg.
+ */
+
+ old = epi->event.events;
+ do {
+ flags = old;
+ } while ((old = cmpxchg(&epi->event.events, flags,
+ flags & EP_PRIVATE_BITS)) != flags);
+
+ return flags & ~EP_PRIVATE_BITS;
+}
AFAICT epi->event.events also has normal writes to it, eg. in
ep_modify(). A number of architectures cannot handle concurrent normal
writes and cmpxchg() to the same variable.
Yes, we race with the current function and with ep_modify(). Then,
ep_modify()
should do something as the following:
- epi->event.events = event->events
+ xchg(&epi->event.events, event->events);
Is that ok?
Just curious: what are these archs?
Thanks.
--
Roman
