Re: instrumentation, was Re: core dump analysis

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Michael,

On 11.4.2023 11.24, Michael Schmitz wrote:
Am 11.04.2023 um 19:19 schrieb Geert Uytterhoeven:
On Tue, Apr 11, 2023 at 6:56 AM Michael Schmitz <schmitzmic@xxxxxxxxx>
Am 11.04.2023 um 12:20 schrieb Finn Thain:
A better solution might be be to port the existing instrumentation like
ftrace, kprobes, uprobes etc. Might be a lot of work though. I wonder
how portable that stuff is.

Another possibility is just using manually added kernel tracepoints:
https://www.kernel.org/doc/html/latest/trace/tracepoints.html

"git grep TRACE_EVENT" did not given any hits for arch/m68k/ though.

(I don't think those need any particular architecture support. One just adds them to locations where it makes sense to trace activity that is important, but which does not happen too often for the very small overhead from disabled static trace point to be significant.)


Looking at a few arch implementations, I'm utterly confused. Wouldn't know where to start.

Good article: https://lwn.net/Articles/132196/

(Ignore jprobes stuff as that's deprecated.)


Because probe replaces instruction at the probe point with a breakpoint:
https://www.kernel.org/doc/html/latest/trace/kprobes.html#how-does-a-kprobe-work

It needs architecture specific code to decode and simulate the replaced instruction (its side-effects) when it is later executed "out-of-line".

This also causes architecture specific limitations on where the probe can be set. Architecture may not support simulating all possible instructions, just the most common ones (e.g. ones typically used on function entry and exit points).

Besides that, AFAIK plain kprobes support should not have much architecture specific stuff.


(I have no experience with kernel code, I just remember some LWN articles about kprobes during past 2 decades, and I was involved with a tool that did something similar in user-space, before Linux got uprobes support.)



As to what architecture to use as an example, I think it would be best to ask tracing maintainers for advice, as there seems to be a lot of code which has been copied from each other at different points in time, that might be nowadays better as shared rather than under each arch...

These architectures support kprobes:
------------------------------------
$ grep " ok " Documentation/features/debug/kprobes/arch-support.txt
    |         arc: |  ok  |
    |         arm: |  ok  |
    |       arm64: |  ok  |
    |        csky: |  ok  |
    |        ia64: |  ok  |
    |        mips: |  ok  |
    |      parisc: |  ok  |
    |     powerpc: |  ok  |
    |       riscv: |  ok  |
    |        s390: |  ok  |
    |          sh: |  ok  |
    |       sparc: |  ok  |
    |         x86: |  ok  |
------------------------------------

But only these seem to have probing functionality isolated nicely to its own directory:
------------------------------------
$ ls arch/*/kernel/probes/

arch/arm64/kernel/probes/:
decode-insn.c decode-insn.h kprobes.c kprobes_trampoline.S Makefile simulate-insn.c simulate-insn.h uprobes.c

arch/csky/kernel/probes/:
decode-insn.c decode-insn.h ftrace.c kprobes.c kprobes_trampoline.S Makefile simulate-insn.c simulate-insn.h uprobes.c

arch/riscv/kernel/probes/:
decode-insn.c decode-insn.h ftrace.c kprobes.c Makefile rethook.c rethook.h rethook_trampoline.S simulate-insn.c simulate-insn.h uprobes.c
------------------------------------


As to testing whether kprobes work, that could be easier with ftrace support also present:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/ftrace/README


	- Eero

PS. Looking at the kernel features list, m68k arch seems to be missing a lot of other features too:
https://www.kernel.org/doc/html/latest/m68k/features.html



[Index of Archives]     [Video for Linux]     [Yosemite News]     [Linux S/390]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux