On Wed, 29 Sep 2021 14:32:37 +0200
Matteo Croce <mcroce@xxxxxxxxxxxxxxxxxxx> wrote:
> On Tue, Sep 28, 2021 at 10:35 PM Alexei Starovoitov
> <alexei.starovoitov@xxxxxxxxx> wrote:
> >
> > On Tue, Sep 28, 2021 at 10:11 AM Matteo Croce
> > <mcroce@xxxxxxxxxxxxxxxxxxx> wrote:
> > >
> > > On Tue, 28 Sep 2021 09:37:30 -0700
> > > Alexei Starovoitov <alexei.starovoitov@xxxxxxxxx> wrote:
> > >
> > > > On Tue, Sep 28, 2021 at 04:45:15PM +0200, Matteo Croce wrote:
> > > > > On Fri, 17 Sep 2021 14:57:12 -0700
> > > > > Alexei Starovoitov <alexei.starovoitov@xxxxxxxxx> wrote:
> > > > >
> > > > > > From: Alexei Starovoitov <ast@xxxxxxxxxx>
> > > > > >
> > > > > > Make relo_core.c to be compiled with kernel and with libbpf.
> > > > > >
> > > > > > Signed-off-by: Alexei Starovoitov <ast@xxxxxxxxxx>
> > > > > > ---
> > > > >
> > > > > I give it a try with a sample co-re program.
> > > >
> > > > Thanks for testing!
> > > >
> > >
> > > I just found an error during the relocations.
> > > It was hiding because of bpf_core_apply_relo() always returning
> > > success[1].
> > >
> > > I have a BPF with the following programs:
> > >
> > > #if 0
> > > SEC("tp_btf/xdp_devmap_xmit")
> > > int BPF_PROG(tp_xdp_devmap_xmit_multi, const struct net_device
> > > *from_dev, const struct net_device *to_dev, int sent, int drops,
> > > int err)
> > >
> > > {
> > > randmap(from_dev->ifindex);
> > > return 0;
> > > }
> > > #endif
> > >
> > > SEC("fentry/eth_type_trans")
> > > int BPF_PROG(fentry_eth_type_trans, struct sk_buff *skb,
> > > struct net_device *dev, unsigned short protocol)
> > > {
> > > randmap(dev->ifindex + skb->len);
> > > return 0;
> > > }
> > >
> > > SEC("fexit/eth_type_trans")
> > > int BPF_PROG(fexit_eth_type_trans, struct sk_buff *skb,
> > > struct net_device *dev, unsigned short protocol)
> > > {
> > > randmap(dev->ifindex + skb->len);
> > > return 0;
> > > }
> > >
> > >
> > > randmap() just writes the value into a random map. If I keep #if 0
> > > everything works, if I remove it so to build
> > > tp_btf/xdp_devmap_xmit too, I get this:
> > >
> > > [ 3619.229378] libbpf: prog 'prog_name': relo #0: kind <byte_off>
> > > (0), spec is [24] STRUCT net_device.ifindex (0:17 @ offset 208) [
> > > 3619.229384] libbpf: prog 'prog_name': relo #0: matching
> > > candidate #0 [2617] STRUCT net_device.ifindex (0:17 @ offset 208)
> > > [ 3619.229538] libbpf: prog 'prog_name': relo #0: patched insn #0
> > > (LDX/ST/STX) off 208 -> 208 [ 3619.230278] libbpf: prog
> > > 'prog_name': relo #0: kind <byte_off> (0), spec is [87] STRUCT
> > > sk_buff.len (0:5 @ offset 104) [ 3619.230282] libbpf: prog
> > > 'prog_name': relo #0: matching candidate #0 [2660] STRUCT
> > > sk_buff.len (0:5 @ offset 104) [ 3619.230393] libbpf: prog
> > > 'prog_name': relo #0: trying to relocate unrecognized insn #0,
> > > code:0x85, src:0x0, dst:0x0, off:0x0, imm:0x7 [ 3619.230562]
> > > libbpf: prog 'prog_name': relo #0: failed to patch insn #0: -22
> > >
> > > The program in tp_btf/xdp_devmap_xmit makes the relocations into
> > > another section fail, note that sk_buff.len is used in the fentry
> > > program.
> > >
> > > Ideas?
> >
> > I'll take a look. Could you provide the full .c file?
>
> Sure. I put everything in this repo:
>
> https://gist.github.com/teknoraver/2855e0f8770d1363b57d683fa32bccc3
>
> tp_btf/xdp_devmap_xmit is the program which lets the other fail.
>
I enabled debugging in userspace libbpf to compare the two outputs.
Userspace libbpf:
libbpf: CO-RE relocating [0] struct net_device: found target candidate [2617] struct net_device in [vmlinux]
libbpf: prog 'tp_xdp_devmap_xmit_multi': relo #0: kind <byte_off> (0), spec is [24] struct net_device.ifindex (0:17 @ offset 208)
libbpf: prog 'tp_xdp_devmap_xmit_multi': relo #0: matching candidate #0 [2617] struct net_device.ifindex (0:17 @ offset 208)
libbpf: prog 'tp_xdp_devmap_xmit_multi': relo #0: patched insn #2 (LDX/ST/STX) off 208 -> 208
libbpf: prog 'tp_xdp_devmap_xmit_multi': relo #1: kind <byte_off> (0), spec is [24] struct net_device.ifindex (0:17 @ offset 208)
libbpf: prog 'tp_xdp_devmap_xmit_multi': relo #1: matching candidate #0 [2617] struct net_device.ifindex (0:17 @ offset 208)
libbpf: prog 'tp_xdp_devmap_xmit_multi': relo #1: patched insn #3 (LDX/ST/STX) off 208 -> 208
libbpf: sec 'fentry/eth_type_trans': found 2 CO-RE relocations
libbpf: CO-RE relocating [0] struct sk_buff: found target candidate [2660] struct sk_buff in [vmlinux]
libbpf: prog 'fentry_eth_type_trans': relo #0: kind <byte_off> (0), spec is [87] struct sk_buff.len (0:5 @ offset 104)
libbpf: prog 'fentry_eth_type_trans': relo #0: matching candidate #0 [2660] struct sk_buff.len (0:5 @ offset 104)
libbpf: prog 'fentry_eth_type_trans': relo #0: patched insn #2 (LDX/ST/STX) off 104 -> 104
libbpf: prog 'fentry_eth_type_trans': relo #1: kind <byte_off> (0), spec is [24] struct net_device.ifindex (0:17 @ offset 208)
libbpf: prog 'fentry_eth_type_trans': relo #1: matching candidate #0 [2617] struct net_device.ifindex (0:17 @ offset 208)
libbpf: prog 'fentry_eth_type_trans': relo #1: patched insn #3 (LDX/ST/STX) off 208 -> 208
libbpf: sec 'fexit/eth_type_trans': found 2 CO-RE relocations
libbpf: prog 'fexit_eth_type_trans': relo #0: kind <byte_off> (0), spec is [87] struct sk_buff.len (0:5 @ offset 104)
libbpf: prog 'fexit_eth_type_trans': relo #0: matching candidate #0 [2660] struct sk_buff.len (0:5 @ offset 104)
libbpf: prog 'fexit_eth_type_trans': relo #0: patched insn #2 (LDX/ST/STX) off 104 -> 104
libbpf: prog 'fexit_eth_type_trans': relo #1: kind <byte_off> (0), spec is [24] struct net_device.ifindex (0:17 @ offset 208)
libbpf: prog 'fexit_eth_type_trans': relo #1: matching candidate #0 [2617] struct net_device.ifindex (0:17 @ offset 208)
libbpf: prog 'fexit_eth_type_trans': relo #1: patched insn #3 (LDX/ST/STX) off 208 -> 208
Kernel implementation
[13234.650397] libbpf: prog 'prog_name': relo #0: kind <byte_off> (0), spec is [24] STRUCT net_device.ifindex (0:17 @ offset 208)
[13234.650406] libbpf: prog 'prog_name': relo #0: matching candidate #0 [2617] STRUCT net_device.ifindex (0:17 @ offset 208)
[13234.650558] libbpf: prog 'prog_name': relo #0: patched insn #0 (LDX/ST/STX) off 208 -> 208
[13234.651251] libbpf: prog 'prog_name': relo #0: kind <byte_off> (0), spec is [24] STRUCT net_device.ifindex (0:17 @ offset 208)
[13234.651255] libbpf: prog 'prog_name': relo #0: matching candidate #0 [2617] STRUCT net_device.ifindex (0:17 @ offset 208)
[13234.651349] libbpf: prog 'prog_name': relo #0: patched insn #0 (LDX/ST/STX) off 208 -> 208
[13234.651935] libbpf: prog 'prog_name': relo #0: kind <byte_off> (0), spec is [87] STRUCT sk_buff.len (0:5 @ offset 104)
[13234.651939] libbpf: prog 'prog_name': relo #0: matching candidate #0 [2660] STRUCT sk_buff.len (0:5 @ offset 104)
[13234.652001] libbpf: prog 'prog_name': relo #0: unexpected insn #0 (LDX/ST/STX) value: got 208, exp 104 -> 104
[13234.652105] libbpf: prog 'prog_name': relo #0: failed to patch insn #0: -22
The sk_buff.len has a wrong offset, 208 instead of 104. Can it be a
leftover value from the previous relocation?
Regards,
--
per aspera ad upstream
