Hi!
After much breaking of things, find here the improved version.
Since v3:
- unbreak everything that does *NOT* have cmpxchg128()
Notably this_cpu_cmpxchg_double() is used unconditionally by SLUB
which means that this_cpu_try_cmpxchg128() needs to be unconditionally
available on all 64bit architectures.
- fixed up x86/x86_64 cmpxchg{8,16}b emulation for this_cpu_cmpxchg{64,128}()
- introduce {raw,this}_cpu_try_cmpxchg*()
- add fallback for !__SIZEOF_INT128__ 64bit architectures
Sadly there are supported 64bit architecture/compiler combinations that do
not have __SIZEOF_INT128__, specifically it was found that HPPA64 only added
this with GCC-11.
this is yuck, and ideally we'd simply raise compiler requirements, but this
'works'.
My plan is to re-add this to tip/locking/core and thus -next later this week.
Also available at:
git://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git locking/core
---
Documentation/core-api/this_cpu_ops.rst | 2 -
arch/arm64/include/asm/atomic_ll_sc.h | 56 +++---
arch/arm64/include/asm/atomic_lse.h | 39 ++---
arch/arm64/include/asm/cmpxchg.h | 48 ++----
arch/arm64/include/asm/percpu.h | 30 ++--
arch/s390/include/asm/cmpxchg.h | 32 +---
arch/s390/include/asm/cpu_mf.h | 2 +-
arch/s390/include/asm/percpu.h | 34 ++--
arch/s390/kernel/perf_cpum_sf.c | 16 +-
arch/x86/include/asm/cmpxchg.h | 25 ---
arch/x86/include/asm/cmpxchg_32.h | 2 +-
arch/x86/include/asm/cmpxchg_64.h | 63 ++++++-
arch/x86/include/asm/percpu.h | 102 ++++++-----
arch/x86/lib/Makefile | 3 +-
arch/x86/lib/cmpxchg16b_emu.S | 43 +++--
arch/x86/lib/cmpxchg8b_emu.S | 67 ++++++--
drivers/iommu/amd/amd_iommu_types.h | 9 +-
drivers/iommu/amd/iommu.c | 10 +-
drivers/iommu/intel/irq_remapping.c | 8 +-
include/asm-generic/percpu.h | 257 ++++++++++++++++++++++------
include/crypto/b128ops.h | 14 +-
include/linux/atomic/atomic-arch-fallback.h | 95 +++++++++-
include/linux/atomic/atomic-instrumented.h | 93 ++++++++--
include/linux/dmar.h | 125 +++++++-------
include/linux/percpu-defs.h | 45 ++---
include/linux/slub_def.h | 12 +-
include/linux/types.h | 12 ++
include/uapi/linux/types.h | 4 +
lib/crypto/curve25519-hacl64.c | 2 -
lib/crypto/poly1305-donna64.c | 2 -
mm/slab.h | 53 +++++-
mm/slub.c | 139 +++++++++------
scripts/atomic/gen-atomic-fallback.sh | 4 +-
scripts/atomic/gen-atomic-instrumented.sh | 19 +-
34 files changed, 952 insertions(+), 515 deletions(-)
