Alexei Starovoitov <alexei.starovoitov@xxxxxxxxx> writes:
> On Fri, Nov 08, 2019 at 09:17:12PM +0100, Toke Høiland-Jørgensen wrote:
>> Alexei Starovoitov <ast@xxxxxxxxxx> writes:
>>
>> > Allow FENTRY/FEXIT BPF programs to attach to other BPF programs of any type
>> > including their subprograms. This feature allows snooping on input and output
>> > packets in XDP, TC programs including their return values. In order to do that
>> > the verifier needs to track types not only of vmlinux, but types of other BPF
>> > programs as well. The verifier also needs to translate uapi/linux/bpf.h types
>> > used by networking programs into kernel internal BTF types used by FENTRY/FEXIT
>> > BPF programs. In some cases LLVM optimizations can remove arguments from BPF
>> > subprograms without adjusting BTF info that LLVM backend knows. When BTF info
>> > disagrees with actual types that the verifiers sees the BPF trampoline has to
>> > fallback to conservative and treat all arguments as u64. The FENTRY/FEXIT
>> > program can still attach to such subprograms, but won't be able to recognize
>> > pointer types like 'struct sk_buff *' into won't be able to pass them to
>> > bpf_skb_output() for dumping to user space.
>> >
>> > The BPF_PROG_LOAD command is extended with attach_prog_fd field. When it's set
>> > to zero the attach_btf_id is one vmlinux BTF type ids. When attach_prog_fd
>> > points to previously loaded BPF program the attach_btf_id is BTF type id of
>> > main function or one of its subprograms.
>> >
>> > Signed-off-by: Alexei Starovoitov <ast@xxxxxxxxxx>
>>
>> This is cool! Certainly solves the xdpdump use case; thanks!
>>
>> I do have a few questions (thinking about whether it can also be used
>> for running multiple XDP programs):
>
> excellent questions.
>
>> - Can a FEXIT function loaded this way only *observe* the return code of
>> the BPF program it attaches to, or can it also change it?
>
> yes. the verifier can be taught to support that for certain class of programs.
> That needs careful thinking to make sure it's safe.
OK. I think this could potentially be useful to have for XDP (for
instance, to have xdpdump "steal" any packets it is observing by
changing the return code to XDP_DROP).
>> - Is it possible to attach multiple FENTRY/FEXIT programs to the same
>> XDP program
>
> Yes. Already possible. See fexit_stress.c that attaches 40 progs to the same
> kernel function. Same thing when attaching fexit BPF to any XDP program.
> Since all of them are read only tracing prog all progs have access to skb on
> input and ouput along with unmodified return value.
Right, cool.
>> and/or to recursively attach FENTRY/FEXIT programs to each
>> other?
>
> Not right now to avoid complex logic of detecting cycles. See simple bit:
> if (tgt_prog->type == BPF_PROG_TYPE_TRACING) {
> /* prevent cycles */
> verbose(env, "Cannot recursively attach\n");
OK, that is probably a reasonable tradeoff.
>> - Could it be possible for an FENTRY/FEXIT program to call into another
>> XDP program (i.e., one that has the regular XDP program type)?
>
> It's possible to teach verifier to do that, but we probably shouldn't take that
> route. Instead I've started exploring the idea of dynamic linking. The
> trampoline logic will be used to replace existing BPF program or subprogram
> instead of attaching read-only to it. If types match the new program can
> replace existing one. The concept allows to build any kind of callchain
> programmatically. Pretty much what Ed proposed with static linking, but doing
> it dynamically. I'll start a separate email thread explaining details.
SGTM; will wait for the sequel, then :)
-Toke
