Re: Trying the bpf trace a bpf xdp program

"Eelco Chaudron" <echaudro@xxxxxxxxxx> · Thu, 05 Dec 2019 13:40:24 +0100

On 4 Dec 2019, at 19:52, Eelco Chaudron wrote:


On 4 Dec 2019, at 19:01, Yonghong Song wrote:

<SNIP>


I’ve put my code on GitHub, maybe it’s just something stupid…


Thanks for the test case. This indeed a kernel bug.
The following change fixed the issue:


-bash-4.4$ git diff

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index a0482e1c4a77..034ef81f935b 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -9636,7 +9636,10 @@ static int check_attach_btf_id(struct
bpf_verifier_env *env)
                                 ret = -EINVAL;
                                 goto out;
                         }
-                       addr = (long)
tgt_prog->aux->func[subprog]->bpf_func;
+                       if (subprog == 0)
+                               addr = (long) tgt_prog->bpf_func;
+                       else
+                               addr = (long)
tgt_prog->aux->func[subprog]->bpf_func;
                 } else {
                         addr = kallsyms_lookup_name(tname);
                         if (!addr) {
-bash-4.4$

The reason is for a bpf program without any additional subprogram

(callees), tgt_prog->aux->func is not populated and is a NULL 

pointer,

so the access tgt_prog->aux->func[0]->bpf_func will segfault.

With the above change, your test works properly.



Thanks for the quick response, and as you mention the test passes with 

the patch above.



I will continue my experiments later this week, and let you know if I 

run into any other problems.




With the following program I get some access errors:

#define bpf_debug(fmt, ...)                         \
{                                                   \
  char __fmt[] = fmt;                               \
  bpf_trace_printk(__fmt, sizeof(__fmt),            \
                   ##__VA_ARGS__);                  \
}

BPF_TRACE_2("fexit/xdp_prog_simple", trace_on_exit,
            struct xdp_md *, xdp, int, ret)
{
  __u32 rx_queue;

  __builtin_preserve_access_index(({
        rx_queue = xdp->rx_queue_index;
      }));

  bpf_debug("fexit: queue = %u, ret = %d\n", rx_queue, ret);

  return 0;
}

I assume the XDP context has not been vetted?

libbpf: -- BEGIN DUMP LOG ---
libbpf:
func#0 @0
BPF program ctx type is not a struct
Type info disagrees with actual arguments due to compiler optimizations
0: R1=ctx(id=0,off=0,imm=0) R10=fp0
; BPF_TRACE_2("fexit/xdp_prog_simple", trace_on_exit,
0: (b7) r2 = 16
1: R1=ctx(id=0,off=0,imm=0) R2_w=inv16 R10=fp0
; BPF_TRACE_2("fexit/xdp_prog_simple", trace_on_exit,
1: (79) r3 = *(u64 *)(r1 +0)

2: R1=ctx(id=0,off=0,imm=0) R2_w=inv16 

R3_w=ptr_xdp_buff(id=0,off=0,imm=0) R10=fp0

2: (0f) r3 += r2
last_idx 2 first_idx 0
regs=4 stack=0 before 1: (79) r3 = *(u64 *)(r1 +0)
regs=4 stack=0 before 0: (b7) r2 = 16

3: R1=ctx(id=0,off=0,imm=0) R2_w=invP16 

R3_w=ptr_xdp_buff(id=0,off=16,imm=0) R10=fp0

; rx_queue = xdp->rx_queue_index;
3: (61) r3 = *(u32 *)(r3 +0)

cannot access ptr member data_meta with moff 16 in struct xdp_buff with 

off 16 size 4

verification time 102 usec
stack depth 0

processed 4 insns (limit 1000000) max_states_per_insn 0 total_states 0 

peak_states 0 mark_read 0

libbpf: -- END LOG --



Trying to use the helpers, passes verification, however, it’s dumping 

invalid content:


BPF_TRACE_2("fexit/xdp_prog_simple", trace_on_exit,
            struct xdp_md *, xdp, int, ret)
{
  __u32 rx_queue;

  bpf_probe_read_kernel(&rx_queue, sizeof(rx_queue),
                        __builtin_preserve_access_index(&xdp->rx_queue_index));

  bpf_debug("fexit: queue = %u, ret = %d\n", rx_queue, ret);
  return 0;
}

Debug output:


   ping6-2752  [004] ..s1 60763.917790: 0: SIMPLE: [ifindex = 4, queue 

=  0]

   ping6-2752  [004] ..s1 60763.917800: 0: fexit: queue = 2969379072, 

ret = 2

   ping6-2752  [004] ..s1 60764.941817: 0: SIMPLE: [ifindex = 4, queue 

=  0]

   ping6-2752  [004] ..s1 60764.941828: 0: fexit: queue = 2969379072, 

ret = 2

   ping6-2752  [004] ..s1 60765.965835: 0: SIMPLE: [ifindex = 4, queue 

=  0]




Tried the same with fentry for this function, but the same results as 

fexit.


Any hints?

Thanks,


Eelco




