The patch titled
Subject: bitops: use try_cmpxchg in set_mask_bits and bit_clear_unless
has been added to the -mm mm-nonmm-unstable branch. Its filename is
bitops-use-try_cmpxchg-in-set_mask_bits-and-bit_clear_unless.patch
This patch will shortly appear at
https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/bitops-use-try_cmpxchg-in-set_mask_bits-and-bit_clear_unless.patch
This patch will later appear in the mm-nonmm-unstable branch at
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
Before you just go and hit "reply", please:
a) Consider who else should be cc'ed
b) Prefer to cc a suitable mailing list as well
c) Ideally: find the original patch on the mailing list and do a
reply-to-all to that, adding suitable additional cc's
*** Remember to use Documentation/process/submit-checklist.rst when testing your code ***
The -mm tree is included into linux-next via the mm-everything
branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
and is updated there every 2-3 working days
------------------------------------------------------
From: Uros Bizjak <ubizjak@xxxxxxxxx>
Subject: bitops: use try_cmpxchg in set_mask_bits and bit_clear_unless
Date: Mon, 22 Aug 2022 16:38:51 +0200
Use try_cmpxchg instead of cmpxchg (*ptr, old, new) == old in
set_mask_bits and bit_clear_unless. x86 CMPXCHG instruction returns
success in ZF flag, so this change saves a compare after cmpxchg (and
related move instruction in front of cmpxchg).
Also, try_cmpxchg implicitly assigns old *ptr value to "old" when cmpxchg
fails, enabling further code simplifications.
Link: https://lkml.kernel.org/r/20220822143851.3290-1-ubizjak@xxxxxxxxx
Signed-off-by: Uros Bizjak <ubizjak@xxxxxxxxx>
Signed-off-by: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
---
include/linux/bitops.h | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
--- a/include/linux/bitops.h~bitops-use-try_cmpxchg-in-set_mask_bits-and-bit_clear_unless
+++ a/include/linux/bitops.h
@@ -327,10 +327,10 @@ static __always_inline void __assign_bit
const typeof(*(ptr)) mask__ = (mask), bits__ = (bits); \
typeof(*(ptr)) old__, new__; \
\
+ old__ = READ_ONCE(*(ptr)); \
do { \
- old__ = READ_ONCE(*(ptr)); \
new__ = (old__ & ~mask__) | bits__; \
- } while (cmpxchg(ptr, old__, new__) != old__); \
+ } while (!try_cmpxchg(ptr, &old__, new__)); \
\
old__; \
})
@@ -342,11 +342,12 @@ static __always_inline void __assign_bit
const typeof(*(ptr)) clear__ = (clear), test__ = (test);\
typeof(*(ptr)) old__, new__; \
\
+ old__ = READ_ONCE(*(ptr)); \
do { \
- old__ = READ_ONCE(*(ptr)); \
+ if (old__ & test__) \
+ break; \
new__ = old__ & ~clear__; \
- } while (!(old__ & test__) && \
- cmpxchg(ptr, old__, new__) != old__); \
+ } while (!try_cmpxchg(ptr, &old__, new__)); \
\
!(old__ & test__); \
})
_
Patches currently in -mm which might be from ubizjak@xxxxxxxxx are
llist-use-try_cmpxchg-in-llist_add_batch-and-llist_del_first.patch
epoll-use-try_cmpxchg-in-list_add_tail_lockless.patch
buffer-use-try_cmpxchg-in-discard_buffer.patch
aio-use-atomic_try_cmpxchg-in-__get_reqs_available.patch
iversion-use-atomic64_try_cmpxchg.patch
bitops-use-try_cmpxchg-in-set_mask_bits-and-bit_clear_unless.patch
