On Wed, Mar 22, 2023 at 6:10 AM Jiri Olsa <olsajiri@xxxxxxxxx> wrote: > > On Wed, Mar 22, 2023 at 08:33:18AM +0100, Davide Miola wrote: > > > seems correct to me, can you see see recursion_misses counter in > > > bpftool prog output for the entry tracing program? > > > > Indeed I can. The problem here is that the recursion is not triggered > > by my programs; from my point of view any miss is basically a random > > event, and the fact that entry and exit progs can miss independently > > means that, at any point, I can count two exits for one entry or > > (worse) just one exit for two entries, making the whole mechanism > > wildly unreliable. > > > > Would using kprobes/kretprobes instead of fentry/fexit here be my > > best compromise? It is my understanding (please correct me if I'm > > wrong) that kprobes' recursion prevention is per-cpu rather than > > per-program, so in this case there would be no imbalance in the > > number of misses between the entry and exit probes. > > kprobes are guarded with perf-cpu bpf_prog_active variable, which will > block nested kprobe calls while kprobe program is running, so you will > get balanced counts but likely miss some executions > > there was discussion about this some time ago: > https://lore.kernel.org/bpf/CAEf4BzZ-xe-zSjbBpKLHfQKPnTRTBMA2Eg382+_4kQoTLnj4eQ@xxxxxxxxxxxxxx/ > > seems the 'active' problem andrii described fits to your case as well > > also retsnoop is probably using some custom per-cpu logic in each program > to prevent this issue, you check it in here: > git@xxxxxxxxxx:anakryiko/retsnoop.git I suspect per-cpu recursion counter will miss more events in this case, since _any_ kprobe on that cpu will be blocked. If missing events is not an issue you probably want a per-cpu counter that is specific to your single ip_queue_xmit attach point. But I'd explore proper networking hooks first that don't have such counter. There are quite a few in TC and cgroup-bpf layer that can be attached to. netfilter-bpf hooks are wip.
