Hello, There have been some requests for low-overhead kernel lock contention monitoring. The kernel has CONFIG_LOCK_STAT to provide such an infra either via /proc/lock_stat or tracepoints directly. However it's not light-weight and hard to be used in production. So I'm trying to add new tracepoints for lock contention and using them as a base to build a new monitoring system. * Changes in v4 - use __print_flags in the TP_printk() - reworked __down_common for semaphore - add Tested-by from Hyeonggon Yoo * Changes in v3 - move the tracepoints deeper in the slow path - remove the caller ip - don't use task state in the flags - add 'ret' field to the contention end tracepoint * Changes in v2 - do not use lockdep infrastructure - add flags argument to lock:contention_begin tracepoint I added a flags argument in the contention_begin to classify locks in question. It can tell whether it's a spinlock, reader-writer lock or a mutex. With stacktrace, users can identify which lock is contended. The patch 01 added the tracepoints and move the definition to the mutex.c file so that it can see the tracepoints without lockdep. The patch 02 actually installs the tracepoints in the locking code. To minimize the overhead, they were added in the slow path of the code separately. As spinlocks are defined in the arch headers, I couldn't handle them all. I've just added it to generic queued spinlock and rwlocks only. Each arch can add the tracepoints later. This series base on the current tip/locking/core and you get it from 'locking/tracepoint-v4' branch in my tree at: git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git Thanks, Namhyung Namhyung Kim (2): locking: Add lock contention tracepoints locking: Apply contention tracepoints in the slow path include/trace/events/lock.h | 61 +++++++++++++++++++++++++++++++++-- kernel/locking/lockdep.c | 1 - kernel/locking/mutex.c | 6 ++++ kernel/locking/percpu-rwsem.c | 3 ++ kernel/locking/qrwlock.c | 9 ++++++ kernel/locking/qspinlock.c | 5 +++ kernel/locking/rtmutex.c | 11 +++++++ kernel/locking/rwbase_rt.c | 3 ++ kernel/locking/rwsem.c | 9 ++++++ kernel/locking/semaphore.c | 15 ++++++++- 10 files changed, 118 insertions(+), 5 deletions(-) base-commit: cd27ccfc727e99352321c0c75012ab9c5a90321e -- 2.35.1.894.gb6a874cedc-goog