Hello, There have been some requests for low-overhead kernel lock contention monitoring. The kernel has CONFIG_LOCK_STAT to provide such an infra either via /proc/lock_stat or tracepoints directly. However it's not light-weight and hard to be used in production. So I'm trying to add new tracepoints for lock contention and using them as a base to build a new monitoring system. * Changes in v2 - do not use lockdep infrastructure - add flags argument to lock:contention_begin tracepoint As we don't want to increase the size of lock data structure, it's hard to have the name of locks or their classes. Instead we can use caller IP to identify contended locks. I had to modify some places to have that information meaningful. Also, I added a flags argument in the contention_begin to classify locks in question. The lower 2 bytes will have a task state it goes to. This can be TASK_RUNNING (0) for spinlocks, or other values for sleeping locks (mutex, rwsem, ...). And the upper 2 bytes will have addition info like whether it's a reader-writer lock or real-time and so on. The patch 01 added the tracepoints in a new file and two new wrapper functions were added. This file contains definition of all locking tracepoints including lockdep/lock_stat. The wrappers are necessary because some kind of locks are defined in a header file and it was not possible to include tracepoint headers directly due to circular dependencies. The patch 02 actually installs the tracepoints in the locking code. To minimize the overhead, they were added in the slow path of the code separately. As spinlocks are defined in the arch headers, I couldn't handle them all. I've just added it to generic queued spinlock and rwlocks only. Each arch can add the tracepoints later. The patch 03 and 04 updates the mutex and rwsem code to pass proper caller IPs. Getting the caller IP at the tracepoint won't work if any of the locking code is not inlined. So pass the IP from the API function to the internal ones. This series base on the current tip/locking/core and you get it from 'locking/tracepoint-v2' branch in my tree at: git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git Thanks, Namhyung Namhyung Kim (4): locking: Add lock contention tracepoints locking: Apply contention tracepoints in the slow path locking/mutex: Pass proper call-site ip locking/rwsem: Pass proper call-site ip include/asm-generic/qrwlock.h | 5 ++ include/asm-generic/qspinlock.h | 3 ++ include/linux/lock_trace.h | 31 +++++++++++++ include/linux/lockdep.h | 29 +++++++++++- include/trace/events/lock.h | 43 ++++++++++++++++- kernel/locking/Makefile | 2 +- kernel/locking/lockdep.c | 1 - kernel/locking/mutex.c | 44 ++++++++++-------- kernel/locking/percpu-rwsem.c | 11 ++++- kernel/locking/rtmutex.c | 12 ++++- kernel/locking/rwbase_rt.c | 11 ++++- kernel/locking/rwsem.c | 81 ++++++++++++++++++++------------- kernel/locking/tracepoint.c | 21 +++++++++ 13 files changed, 235 insertions(+), 59 deletions(-) create mode 100644 include/linux/lock_trace.h create mode 100644 kernel/locking/tracepoint.c base-commit: cd27ccfc727e99352321c0c75012ab9c5a90321e -- 2.35.1.574.g5d30c73bfb-goog
