Hi Milian, On Fri, Apr 22, 2022 at 3:21 AM Milian Wolff <milian.wolff@xxxxxxxx> wrote: > > On Freitag, 22. April 2022 07:33:57 CEST Namhyung Kim wrote: > > Hello, > > > > This is the first version of off-cpu profiling support. Together with > > (PMU-based) cpu profiling, it can show holistic view of the performance > > characteristics of your application or system. > > Hey Namhyung, > > this is awesome news! In hotspot, I've long done off-cpu profiling manually by > looking at the time between --switch-events. The downside is that we also need > to track the sched:sched_switch event to get a call stack. But this approach > also works with dwarf based unwinding, and also includes kernel stacks. Thanks, I've also briefly thought about the switch event based off-cpu profiling as it doesn't require root. But collecting call stacks is hard and I'd like to do it in kernel/bpf to reduce the overhead. > > > With BPF, it can aggregate scheduling stats for interested tasks > > and/or states and convert the data into a form of perf sample records. > > I chose the bpf-output event which is a software event supposed to be > > consumed by BPF programs and renamed it as "offcpu-time". So it > > requires no change on the perf report side except for setting sample > > types of bpf-output event. > > > > Basically it collects userspace callstack for tasks as it's what users > > want mostly. Maybe we can add support for the kernel stacks but I'm > > afraid that it'd cause more overhead. So the offcpu-time event will > > always have callchains regardless of the command line option, and it > > enables the children mode in perf report by default. > > Has anything changed wrt perf/bpf and user applications not compiled with `- > fno-omit-frame-pointer`? I.e. does this new utility only work for specially > compiled applications, or do we also get backtraces for "normal" binaries that > we can install through package managers? I am not aware of such changes, it still needs a frame pointer to get backtraces. Thanks, Namhyung