"Brian G. Merrell" <brian.g.merrell@xxxxxxxxx> writes: > Hi, > > I've been playing around with the new xdp-loader to chain xdp programs > with kernel.bpf_stats_enabled set. This is what I see (from bpftool > prog): > > 22: xdp name xdp_dispatcher tag d51e469e988d81da gpl run_time_ns 105086 run_cnt 24 > loaded_at 2020-12-15T07:52:27+0000 uid 0 > xlated 720B jited 535B memlock 4096B map_ids 5 > btf_id 8 > 24: ext name xdp_pass tag 3b185187f1855c4c gpl > loaded_at 2020-12-15T07:52:27+0000 uid 0 > xlated 16B jited 18B memlock 4096B > btf_id 9 > 26: ext name xdp_drop tag 57cd311f2e27366b gpl > loaded_at 2020-12-15T07:52:27+0000 uid 0 > xlated 16B jited 18B memlock 4096B > btf_id 10 > > You'll notice that the run_time_ns and run_cnt stats are only populated > for the xdp_dispatcher. This is basically the same behavior we see today > with bpf_tail_call chaining, so it's not a regression or anything. It > would be really nice, though, if we could see those stats for each > individual xdp program in the chain. Does anyone know what it would take > for that to happen? The stats are computed by the bits behind the static_branch_unlikely() calls here: https://elixir.bootlin.com/linux/latest/source/kernel/bpf/trampoline.c#L391 The calls to __bpf_prog_enter() and __bpf_prog_exit() is emitted in the top-level dispatcher code that invokes a BPF program from whatever kernel hook it runs at. Which is why it only shows up for the dispatcher: From the kernel PoV that's the top-level BPF program being run, the other programs are patched in as substitutions for function calls inside the top-level BPF program. That being said, the trampoline code that does the function-level patching could conceivably add the same conditional stats update code before and after each jump. Not sure how complicated it would be to add this, but it would mean that stats would automatically appear for each freplace program. > Would it be more feasible for xdplib to optionally report separate > runtime stats itself for chained programs? The dispatcher program could certainly track this by just calling bpf_ktime_get_ns() before and after each sub-program and sticking the results in a map. It wouldn't be reported by the kernel and bpftool, but libxdp and xdp-tools could expose it. The drawback with this approach is that it would be tied to a particular dispatcher instance, so either they would be reset whenever you load an additional program, or the library has to copy over the old stats, which would probably not be doable atomically. For stats that may not matter too much, but it's something to consider. -Toke
|