Re: bpftool gen object doesn't handle GCC built BPF ELF files

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Sat, Jul 9, 2022 at 4:41 PM Jose E. Marchesi
<jose.marchesi@xxxxxxxxxx> wrote:
>
>
> > On Sat, Jul 9, 2022 at 2:32 PM James Hilliard <james.hilliard1@xxxxxxxxx> wrote:
> >>
> >> On Sat, Jul 9, 2022 at 2:21 PM Jose E. Marchesi
> >> <jose.marchesi@xxxxxxxxxx> wrote:
> >> >
> >> >
> >> > > On Sat, Jul 9, 2022 at 11:24 AM Jose E. Marchesi
> >> > > <jose.marchesi@xxxxxxxxxx> wrote:
> >> > >>
> >> > >>
> >> > >> > On Fri, Jul 8, 2022 at 12:33 PM Jose E. Marchesi
> >> > >> > <jose.marchesi@xxxxxxxxxx> wrote:
> >> > >> >>
> >> > >> >>
> >> > >> >> >> On Wed, Jul 6, 2022 at 10:13 AM James Hilliard
> >> > >> >> >> <james.hilliard1@xxxxxxxxx> wrote:
> >> > >> >> >>>
> >> > >> >> >>> Note I'm testing with the following patches:
> >> > >> >> >>> https://lore.kernel.org/bpf/20220706111839.1247911-1-james.hilliard1@xxxxxxxxx/
> >> > >> >> >>> https://lore.kernel.org/bpf/20220706140623.2917858-1-james.hilliard1@xxxxxxxxx/
> >> > >> >> >>>
> >> > >> >> >>> It would appear there's some compatibility issues with bpftool gen and
> >> > >> >> >>> GCC, not sure what side though is wrong here:
> >> > >> >> >>> /home/buildroot/buildroot/output/per-package/systemd/host/sbin/bpftool
> >> > >> >> >>> gen object src/core/bpf/restrict_ifaces/restrict-ifaces.bpf.o
> >> > >> >> >>> src/core/bpf/restrict_ifaces/restrict-ifaces.bpf.unstripped.o
> >> > >> >> >>> libbpf: failed to find BTF info for global/extern symbol 'sd_restrictif_i'
> >> > >> >> >>> Error: failed to link
> >> > >> >> >>> 'src/core/bpf/restrict_ifaces/restrict-ifaces.bpf.unstripped.o':
> >> > >> >> >>> Unknown error -2 (-2)
> >> > >> >> >>>
> >> > >> >> >>> Relevant difference seems to be this:
> >> > >> >> >>> GCC:
> >> > >> >> >>> [55] FUNC 'sd_restrictif_i' type_id=47 linkage=static
> >> > >> >> >>> Clang:
> >> > >> >> >>> [27] FUNC 'sd_restrictif_i' type_id=26 linkage=global
> >> > >> >> >
> >> > >> >> > For functions GCC generates a BTF_KIND_FUNC entry, which has no linkage
> >> > >> >> > information, or so we thought: I just looked at bpftool/btf.c and I
> >> > >> >> > found the linkage info for function types is expected to be encoded in
> >> > >> >> > the vlen field of BTF_KIND_FUNC entries (why not adding a btf_func
> >> > >> >> > instead???) which is surprising to say the least.
> >> > >> >> >
> >> > >> >> > We are changing GCC to encode the linkage info in vlen for these types.
> >> > >> >> > Thanks for reporting this.
> >> > >> >>
> >> > >> >> Patch sent to GCC upstream:
> >> > >> >> https://gcc.gnu.org/pipermail/gcc-patches/2022-July/598090.html
> >> > >> >
> >> > >> > I applied that patch on top of GCC 12.1.0 and it appears to fix the
> >> > >> > bpftool gen object bug.
> >> > >> >
> >> > >> > I am however now hitting a different error during skeleton generation:
> >> > >> > /home/buildroot/buildroot/output/per-package/systemd/host/sbin/bpftool
> >> > >> > gen skeleton src/core/bpf/restrict_ifaces/restrict-ifaces.bpf.o
> >> > >> > libbpf: elf: skipping unrecognized data section(9) .comment
> >> > >> > libbpf: failed to alloc map 'restrict.bss' content buffer: -22
> >> > >> > Error: failed to open BPF object file: Invalid argument
> >> > >>
> >> > >> What is the size of the .bss section in the object file?  Try with:
> >> > >>
> >> > >> $ size restrict-ifaces.bpf.o
> >> > >
> >> > > $ size
> > output/build/systemd-custom/build/src/core/bpf/restrict_ifaces/restrict-ifaces.bpf.o
> >> > >    text       data        bss        dec        hex    filename
> >> > >     386         25          0        411        19b
> >> > > output/build/systemd-custom/build/src/core/bpf/restrict_ifaces/restrict-ifaces.bpf.o
> >> >
> >> > Right, so the .bss section is empty.  I see a `const volatile unsigned
> >> > char is_allow_list = 0;' in restrict-ifaces.bpf.c, but that goes to
> >> > .data and not to .bss, as expected.
> >> >
> >> > If you build restrict-ifaces.bpf.o with LLVM, is the bss still empty?  I
> >> > don't think the code in libbpf.c even checks for this eventuality...
> >>
> >> LLVM version(which skeleton generation works with):
> >> $ size restrict-ifaces.bpf.o
> >>    text       data        bss        dec        hex    filename
> >>     323         24          0        347        15b    restrict-ifaces.bpf.o
> >>
> >> $ /home/buildroot/buildroot/output/per-package/systemd/host/sbin/bpftool
> >> btf dump file restrict-ifaces.bpf.o format raw
> >> [1] PTR '(anon)' type_id=3
> >> [2] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
> >> [3] ARRAY '(anon)' type_id=2 index_type_id=4 nr_elems=1
> >> [4] INT '__ARRAY_SIZE_TYPE__' size=4 bits_offset=0 nr_bits=32 encoding=(none)
> >> [5] PTR '(anon)' type_id=6
> >> [6] TYPEDEF '__u32' type_id=7
> >> [7] INT 'unsigned int' size=4 bits_offset=0 nr_bits=32 encoding=(none)
> >> [8] PTR '(anon)' type_id=9
> >> [9] TYPEDEF '__u8' type_id=10
> >> [10] INT 'unsigned char' size=1 bits_offset=0 nr_bits=8 encoding=(none)
> >> [11] STRUCT '(anon)' size=24 vlen=3
> >>     'type' type_id=1 bits_offset=0
> >>     'key' type_id=5 bits_offset=64
> >>     'value' type_id=8 bits_offset=128
> >> [12] VAR 'sd_restrictif' type_id=11, linkage=global
> >> [13] PTR '(anon)' type_id=14
> >> [14] CONST '(anon)' type_id=15
> >> [15] STRUCT '__sk_buff' size=192 vlen=33
> >>     'len' type_id=6 bits_offset=0
> >>     'pkt_type' type_id=6 bits_offset=32
> >>     'mark' type_id=6 bits_offset=64
> >>     'queue_mapping' type_id=6 bits_offset=96
> >>     'protocol' type_id=6 bits_offset=128
> >>     'vlan_present' type_id=6 bits_offset=160
> >>     'vlan_tci' type_id=6 bits_offset=192
> >>     'vlan_proto' type_id=6 bits_offset=224
> >>     'priority' type_id=6 bits_offset=256
> >>     'ingress_ifindex' type_id=6 bits_offset=288
> >>     'ifindex' type_id=6 bits_offset=320
> >>     'tc_index' type_id=6 bits_offset=352
> >>     'cb' type_id=16 bits_offset=384
> >>     'hash' type_id=6 bits_offset=544
> >>     'tc_classid' type_id=6 bits_offset=576
> >>     'data' type_id=6 bits_offset=608
> >>     'data_end' type_id=6 bits_offset=640
> >>     'napi_id' type_id=6 bits_offset=672
> >>     'family' type_id=6 bits_offset=704
> >>     'remote_ip4' type_id=6 bits_offset=736
> >>     'local_ip4' type_id=6 bits_offset=768
> >>     'remote_ip6' type_id=17 bits_offset=800
> >>     'local_ip6' type_id=17 bits_offset=928
> >>     'remote_port' type_id=6 bits_offset=1056
> >>     'local_port' type_id=6 bits_offset=1088
> >>     'data_meta' type_id=6 bits_offset=1120
> >>     '(anon)' type_id=18 bits_offset=1152
> >>     'tstamp' type_id=20 bits_offset=1216
> >>     'wire_len' type_id=6 bits_offset=1280
> >>     'gso_segs' type_id=6 bits_offset=1312
> >>     '(anon)' type_id=22 bits_offset=1344
> >>     'gso_size' type_id=6 bits_offset=1408
> >>     'hwtstamp' type_id=20 bits_offset=1472
> >> [16] ARRAY '(anon)' type_id=6 index_type_id=4 nr_elems=5
> >> [17] ARRAY '(anon)' type_id=6 index_type_id=4 nr_elems=4
> >> [18] UNION '(anon)' size=8 vlen=1
> >>     'flow_keys' type_id=19 bits_offset=0
> >> [19] PTR '(anon)' type_id=34
> >> [20] TYPEDEF '__u64' type_id=21
> >> [21] INT 'unsigned long long' size=8 bits_offset=0 nr_bits=64 encoding=(none)
> >> [22] UNION '(anon)' size=8 vlen=1
> >>     'sk' type_id=23 bits_offset=0
> >> [23] PTR '(anon)' type_id=35
> >> [24] FUNC_PROTO '(anon)' ret_type_id=2 vlen=1
> >>     'sk' type_id=13
> >> [25] FUNC 'sd_restrictif_e' type_id=24 linkage=global
> >> [26] FUNC 'sd_restrictif_i' type_id=24 linkage=global
> >> [27] CONST '(anon)' type_id=28
> >> [28] VOLATILE '(anon)' type_id=9
> >> [29] VAR 'is_allow_list' type_id=27, linkage=global
> >> [30] CONST '(anon)' type_id=31
> >> [31] INT 'char' size=1 bits_offset=0 nr_bits=8 encoding=SIGNED
> >> [32] ARRAY '(anon)' type_id=30 index_type_id=4 nr_elems=18
> >> [33] VAR '_license' type_id=32, linkage=static
> >> [34] FWD 'bpf_flow_keys' fwd_kind=struct
> >> [35] FWD 'bpf_sock' fwd_kind=struct
> >> [36] DATASEC '.rodata' size=1 vlen=1
> >>     type_id=29 offset=0 size=1 (VAR 'is_allow_list')
> >> [37] DATASEC 'license' size=18 vlen=1
> >>     type_id=33 offset=0 size=18 (VAR '_license')
> >> [38] DATASEC '.maps' size=24 vlen=1
> >>     type_id=12 offset=0 size=24 (VAR 'sd_restrictif')
> >>
> >
> > Skeleton generation debug output for GCC(failing) and LLVM(working)
> > which may be helpful:
>
> Indeed it was helpful :)
>
> The GNU assembler generates an empty .bss section.  This is a well
> established behavior in GAS that happens in all supported targets.
>
> The LLVM assembler doesn't generate an empty .bss section.
>
> bpftool chokes on the empty .bss section.
>
> In this case I would suggest to fix bpf_object__init_global_data_maps in
> order to skip empty sections.
>
> Something like this:

Hmm, that seems to segfault:
Starting program:
/home/buildroot/buildroot/output/per-package/systemd/host/sbin/bpftool
--debug gen skeleton
output/build/systemd-custom/build/src/core/bpf/restrict_ifaces/restrict-ifaces.bpf.o
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
libbpf: loading object 'restrict_ifaces_bpf' from buffer
libbpf: elf: section(2) .symtab, size 336, link 1, flags 0, type=2
libbpf: elf: section(3) .data, size 1, link 0, flags 3, type=1
libbpf: elf: section(4) .bss, size 0, link 0, flags 3, type=8
libbpf: elf: section(5) cgroup_skb/egress, size 184, link 0, flags 6, type=1
libbpf: sec 'cgroup_skb/egress': found program 'sd_restrictif_e' at
insn offset 0 (0 bytes), code size 23 insns (184 bytes)
libbpf: elf: section(6) cgroup_skb/ingress, size 184, link 0, flags 6, type=1
libbpf: sec 'cgroup_skb/ingress': found program 'sd_restrictif_i' at
insn offset 0 (0 bytes), code size 23 insns (184 bytes)
libbpf: elf: section(7) license, size 18, link 0, flags 2, type=1
libbpf: license of restrict_ifaces_bpf is LGPL-2.1-or-later
libbpf: elf: section(8) .maps, size 24, link 0, flags 3, type=1
libbpf: elf: section(9) .comment, size 49, link 0, flags 30, type=1
libbpf: elf: skipping unrecognized data section(9) .comment
libbpf: elf: section(10) .relcgroup_skb/egress, size 32, link 2, flags
40, type=9
libbpf: elf: section(11) .relcgroup_skb/ingress, size 32, link 2,
flags 40, type=9
libbpf: elf: section(12) .BTF, size 3606, link 0, flags 0, type=1
libbpf: looking for externs among 14 symbols...
libbpf: collected 0 externs total
libbpf: map 'sd_restrictif': at sec_idx 8, offset 0.
libbpf: map 'sd_restrictif': found type = 1.
libbpf: map 'sd_restrictif': found key [12], sz = 4.
libbpf: map 'sd_restrictif': found value [3], sz = 1.

Program received signal SIGSEGV, Segmentation fault.
0x0000aaaaaab4fd2c in bpf_object.init_maps ()
(gdb) bt
#0  0x0000aaaaaab4fd2c in bpf_object.init_maps ()
#1  0x0000aaaaaab52178 in bpf_object_open.part ()
#2  0x0000aaaaaab544e8 in bpf_object.open_mem ()
#3  0x0000aaaaaab2a58c in do_skeleton ()
#4  0x0000aaaaaab1e204 in main ()

>
> diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
> index e89cc9c885b3..e3a6808f0bb6 100644
> --- a/tools/lib/bpf/libbpf.c
> +++ b/tools/lib/bpf/libbpf.c
> @@ -1591,6 +1591,10 @@ static int bpf_object__init_global_data_maps(struct bpf_object *obj)
>         for (sec_idx = 1; sec_idx < obj->efile.sec_cnt; sec_idx++) {
>                 sec_desc = &obj->efile.secs[sec_idx];
>
> +                /* Skip empty sections.  */
> +                if (sec_desc->data->d_size == 0)
> +                  continue;
> +
>                 switch (sec_desc->sec_type) {
>                 case SEC_DATA:
>                         sec_name = elf_sec_name(obj, elf_sec_by_idx(obj, sec_idx));
>
> > GCC:
> > $ /home/buildroot/buildroot/output/per-package/systemd/host/sbin/bpftool
> > --debug gen skeleton
> > output/build/systemd-custom/build/src/core/bpf/restrict_ifaces/restrict-ifaces.bpf.o
> > libbpf: loading object 'restrict_ifaces_bpf' from buffer
> > libbpf: elf: section(2) .symtab, size 336, link 1, flags 0, type=2
> > libbpf: elf: section(3) .data, size 1, link 0, flags 3, type=1
> > libbpf: elf: section(4) .bss, size 0, link 0, flags 3, type=8
> > libbpf: elf: section(5) cgroup_skb/egress, size 184, link 0, flags 6, type=1
> > libbpf: sec 'cgroup_skb/egress': found program 'sd_restrictif_e' at
> > insn offset 0 (0 bytes), code size 23 insns (184 bytes)
> > libbpf: elf: section(6) cgroup_skb/ingress, size 184, link 0, flags 6, type=1
> > libbpf: sec 'cgroup_skb/ingress': found program 'sd_restrictif_i' at
> > insn offset 0 (0 bytes), code size 23 insns (184 bytes)
> > libbpf: elf: section(7) license, size 18, link 0, flags 2, type=1
> > libbpf: license of restrict_ifaces_bpf is LGPL-2.1-or-later
> > libbpf: elf: section(8) .maps, size 24, link 0, flags 3, type=1
> > libbpf: elf: section(9) .comment, size 49, link 0, flags 30, type=1
> > libbpf: elf: skipping unrecognized data section(9) .comment
> > libbpf: elf: section(10) .relcgroup_skb/egress, size 32, link 2, flags
> > 40, type=9
> > libbpf: elf: section(11) .relcgroup_skb/ingress, size 32, link 2,
> > flags 40, type=9
> > libbpf: elf: section(12) .BTF, size 3606, link 0, flags 0, type=1
> > libbpf: looking for externs among 14 symbols...
> > libbpf: collected 0 externs total
> > libbpf: map 'sd_restrictif': at sec_idx 8, offset 0.
> > libbpf: map 'sd_restrictif': found type = 1.
> > libbpf: map 'sd_restrictif': found key [12], sz = 4.
> > libbpf: map 'sd_restrictif': found value [3], sz = 1.
> > libbpf: map 'restrict.data' (global data): at sec_idx 3, offset 0, flags 400.
> > libbpf: map 1 is "restrict.data"
> > libbpf: map 'restrict.bss' (global data): at sec_idx 4, offset 0, flags 400.
> > libbpf: failed to alloc map 'restrict.bss' content buffer: -22
> > Error: failed to open BPF object file: Invalid argument
> >
> > LLVM:
> > $ /home/buildroot/buildroot/output/per-package/systemd/host/sbin/bpftool
> > --debug gen skeleton restrict-ifaces.bpf.o
> > libbpf: loading object 'restrict_ifaces_bpf' from buffer
> > libbpf: elf: section(2) .symtab, size 384, link 1, flags 0, type=2
> > libbpf: elf: section(3) cgroup_skb/egress, size 152, link 0, flags 6, type=1
> > libbpf: sec 'cgroup_skb/egress': found program 'sd_restrictif_e' at
> > insn offset 0 (0 bytes), code size 19 insns (152 bytes)
> > libbpf: elf: section(4) cgroup_skb/ingress, size 152, link 0, flags 6, type=1
> > libbpf: sec 'cgroup_skb/ingress': found program 'sd_restrictif_i' at
> > insn offset 0 (0 bytes), code size 19 insns (152 bytes)
> > libbpf: elf: section(5) .rodata, size 1, link 0, flags 2, type=1
> > libbpf: elf: section(6) license, size 18, link 0, flags 2, type=1
> > libbpf: license of restrict_ifaces_bpf is LGPL-2.1-or-later
> > libbpf: elf: section(7) .maps, size 24, link 0, flags 3, type=1
> > libbpf: elf: section(8) .relcgroup_skb/egress, size 32, link 2, flags 40, type=9
> > libbpf: elf: section(9) .relcgroup_skb/ingress, size 32, link 2, flags
> > 40, type=9
> > libbpf: elf: section(10) .BTF, size 1988, link 0, flags 0, type=1
> > libbpf: elf: section(11) .BTF.ext, size 376, link 0, flags 0, type=1
> > libbpf: looking for externs among 16 symbols...
> > libbpf: collected 0 externs total
> > libbpf: map 'sd_restrictif': at sec_idx 7, offset 0.
> > libbpf: map 'sd_restrictif': found type = 1.
> > libbpf: map 'sd_restrictif': found key [6], sz = 4.
> > libbpf: map 'sd_restrictif': found value [9], sz = 1.
> > libbpf: map 'restrict.rodata' (global data): at sec_idx 5, offset 0, flags 480.
> > libbpf: map 1 is "restrict.rodata"
> > libbpf: sec '.relcgroup_skb/egress': collecting relocation for
> > section(3) 'cgroup_skb/egress'
> > libbpf: sec '.relcgroup_skb/egress': relo #0: insn #4 against 'sd_restrictif'
> > libbpf: prog 'sd_restrictif_e': found map 0 (sd_restrictif, sec 7, off
> > 0) for insn #4
> > libbpf: sec '.relcgroup_skb/egress': relo #1: insn #7 against 'is_allow_list'
> > libbpf: prog 'sd_restrictif_e': found data map 1 (restrict.rodata, sec
> > 5, off 0) for insn 7
> > libbpf: sec '.relcgroup_skb/ingress': collecting relocation for
> > section(4) 'cgroup_skb/ingress'
> > libbpf: sec '.relcgroup_skb/ingress': relo #0: insn #4 against 'sd_restrictif'
> > libbpf: prog 'sd_restrictif_i': found map 0 (sd_restrictif, sec 7, off
> > 0) for insn #4
> > libbpf: sec '.relcgroup_skb/ingress': relo #1: insn #7 against 'is_allow_list'
> > libbpf: prog 'sd_restrictif_i': found data map 1 (restrict.rodata, sec
> > 5, off 0) for insn 7
> >
> >> >
> >> > >>
> >> > >> Looking at libbpf.c, it seems to me that this may be due of trying to
> >> > >> mmap 0 bytes in `bpf_object__init_internal_map':
> >> > >>
> >> > >>         map->mmaped = mmap(NULL, bpf_map_mmap_sz(map), PROT_READ | PROT_WRITE,
> >> > >>                            MAP_SHARED | MAP_ANONYMOUS, -1, 0);
> >> > >>         if (map->mmaped == MAP_FAILED) {
> >> > >>                 err = -errno;
> >> > >>                 map->mmaped = NULL;
> >> > >>                 pr_warn("failed to alloc map '%s' content buffer: %d\n",
> >> > >>                         map->name, err);
> >> > >>                 zfree(&map->real_name);
> >> > >>                 zfree(&map->name);
> >> > >>                 return err;
> >> > >>         }
> >> > >>
> >> > >> I see no check for zero sized sections in
> >> > >> bpf_object__init_global_data_maps.
> >> > >>
> >> > >> Is maybe GCC failing to allocate stuff in BSS that is supposed to be
> >> > >> there?
> >> > >>
> >> > >> > Stripped file passed to gen skeleton:
> >> > >> > /home/buildroot/buildroot/output/per-package/systemd/host/sbin/bpftool
> >> > >> > btf dump file
> >> > >> > output/build/systemd-custom/build/src/core/bpf/restrict_ifaces/restrict-ifaces.bpf.o
> >> > >> > format raw
> >> > >> > [1] INT 'signed char' size=1 bits_offset=0 nr_bits=8 encoding=UNKN
> >> > >> > [2] INT 'unsigned char' size=1 bits_offset=0 nr_bits=8 encoding=CHAR
> >> > >> > [3] TYPEDEF '__u8' type_id=2
> >> > >> > [4] CONST '(anon)' type_id=3
> >> > >> > [5] VOLATILE '(anon)' type_id=4
> >> > >> > [6] INT 'short int' size=2 bits_offset=0 nr_bits=16 encoding=SIGNED
> >> > >> > [7] INT 'short unsigned int' size=2 bits_offset=0 nr_bits=16 encoding=(none)
> >> > >> > [8] TYPEDEF '__u16' type_id=7
> >> > >> > [9] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
> >> > >> > [10] TYPEDEF '__s32' type_id=9
> >> > >> > [11] INT 'unsigned int' size=4 bits_offset=0 nr_bits=32 encoding=(none)
> >> > >> > [12] TYPEDEF '__u32' type_id=11
> >> > >> > [13] INT 'long long int' size=8 bits_offset=0 nr_bits=64 encoding=SIGNED
> >> > >> > [14] INT 'long long unsigned int' size=8 bits_offset=0 nr_bits=64
> >> > >> > encoding=(none)
> >> > >> > [15] TYPEDEF '__u64' type_id=14
> >> > >> > [16] INT 'long unsigned int' size=8 bits_offset=0 nr_bits=64 encoding=(none)
> >> > >> > [17] INT 'long int' size=8 bits_offset=0 nr_bits=64 encoding=SIGNED
> >> > >> > [18] INT 'char' size=1 bits_offset=0 nr_bits=8 encoding=UNKN
> >> > >> > [19] CONST '(anon)' type_id=18
> >> > >> > [20] TYPEDEF '__be16' type_id=8
> >> > >> > [21] TYPEDEF '__be32' type_id=12
> >> > >> > [22] ENUM 'bpf_map_type' encoding=UNSIGNED size=4 vlen=31
> >> > >> >     'BPF_MAP_TYPE_UNSPEC' val=0
> >> > >> >     'BPF_MAP_TYPE_HASH' val=1
> >> > >> >     'BPF_MAP_TYPE_ARRAY' val=2
> >> > >> >     'BPF_MAP_TYPE_PROG_ARRAY' val=3
> >> > >> >     'BPF_MAP_TYPE_PERF_EVENT_ARRAY' val=4
> >> > >> >     'BPF_MAP_TYPE_PERCPU_HASH' val=5
> >> > >> >     'BPF_MAP_TYPE_PERCPU_ARRAY' val=6
> >> > >> >     'BPF_MAP_TYPE_STACK_TRACE' val=7
> >> > >> >     'BPF_MAP_TYPE_CGROUP_ARRAY' val=8
> >> > >> >     'BPF_MAP_TYPE_LRU_HASH' val=9
> >> > >> >     'BPF_MAP_TYPE_LRU_PERCPU_HASH' val=10
> >> > >> >     'BPF_MAP_TYPE_LPM_TRIE' val=11
> >> > >> >     'BPF_MAP_TYPE_ARRAY_OF_MAPS' val=12
> >> > >> >     'BPF_MAP_TYPE_HASH_OF_MAPS' val=13
> >> > >> >     'BPF_MAP_TYPE_DEVMAP' val=14
> >> > >> >     'BPF_MAP_TYPE_SOCKMAP' val=15
> >> > >> >     'BPF_MAP_TYPE_CPUMAP' val=16
> >> > >> >     'BPF_MAP_TYPE_XSKMAP' val=17
> >> > >> >     'BPF_MAP_TYPE_SOCKHASH' val=18
> >> > >> >     'BPF_MAP_TYPE_CGROUP_STORAGE' val=19
> >> > >> >     'BPF_MAP_TYPE_REUSEPORT_SOCKARRAY' val=20
> >> > >> >     'BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE' val=21
> >> > >> >     'BPF_MAP_TYPE_QUEUE' val=22
> >> > >> >     'BPF_MAP_TYPE_STACK' val=23
> >> > >> >     'BPF_MAP_TYPE_SK_STORAGE' val=24
> >> > >> >     'BPF_MAP_TYPE_DEVMAP_HASH' val=25
> >> > >> >     'BPF_MAP_TYPE_STRUCT_OPS' val=26
> >> > >> >     'BPF_MAP_TYPE_RINGBUF' val=27
> >> > >> >     'BPF_MAP_TYPE_INODE_STORAGE' val=28
> >> > >> >     'BPF_MAP_TYPE_TASK_STORAGE' val=29
> >> > >> >     'BPF_MAP_TYPE_BLOOM_FILTER' val=30
> >> > >> > [23] UNION '(anon)' size=8 vlen=1
> >> > >> >     'flow_keys' type_id=29 bits_offset=0
> >> > >> > [24] STRUCT 'bpf_flow_keys' size=56 vlen=13
> >> > >> >     'nhoff' type_id=8 bits_offset=0
> >> > >> >     'thoff' type_id=8 bits_offset=16
> >> > >> >     'addr_proto' type_id=8 bits_offset=32
> >> > >> >     'is_frag' type_id=3 bits_offset=48
> >> > >> >     'is_first_frag' type_id=3 bits_offset=56
> >> > >> >     'is_encap' type_id=3 bits_offset=64
> >> > >> >     'ip_proto' type_id=3 bits_offset=72
> >> > >> >     'n_proto' type_id=20 bits_offset=80
> >> > >> >     'sport' type_id=20 bits_offset=96
> >> > >> >     'dport' type_id=20 bits_offset=112
> >> > >> >     '(anon)' type_id=25 bits_offset=128
> >> > >> >     'flags' type_id=12 bits_offset=384
> >> > >> >     'flow_label' type_id=21 bits_offset=416
> >> > >> > [25] UNION '(anon)' size=32 vlen=2
> >> > >> >     '(anon)' type_id=26 bits_offset=0
> >> > >> >     '(anon)' type_id=27 bits_offset=0
> >> > >> > [26] STRUCT '(anon)' size=8 vlen=2
> >> > >> >     'ipv4_src' type_id=21 bits_offset=0
> >> > >> >     'ipv4_dst' type_id=21 bits_offset=32
> >> > >> > [27] STRUCT '(anon)' size=32 vlen=2
> >> > >> >     'ipv6_src' type_id=28 bits_offset=0
> >> > >> >     'ipv6_dst' type_id=28 bits_offset=128
> >> > >> > [28] ARRAY '(anon)' type_id=12 index_type_id=16 nr_elems=4
> >> > >> > [29] PTR '(anon)' type_id=24
> >> > >> > [30] UNION '(anon)' size=8 vlen=1
> >> > >> >     'sk' type_id=32 bits_offset=0
> >> > >> > [31] STRUCT 'bpf_sock' size=80 vlen=14
> >> > >> >     'bound_dev_if' type_id=12 bits_offset=0
> >> > >> >     'family' type_id=12 bits_offset=32
> >> > >> >     'type' type_id=12 bits_offset=64
> >> > >> >     'protocol' type_id=12 bits_offset=96
> >> > >> >     'mark' type_id=12 bits_offset=128
> >> > >> >     'priority' type_id=12 bits_offset=160
> >> > >> >     'src_ip4' type_id=12 bits_offset=192
> >> > >> >     'src_ip6' type_id=28 bits_offset=224
> >> > >> >     'src_port' type_id=12 bits_offset=352
> >> > >> >     'dst_port' type_id=20 bits_offset=384
> >> > >> >     'dst_ip4' type_id=12 bits_offset=416
> >> > >> >     'dst_ip6' type_id=28 bits_offset=448
> >> > >> >     'state' type_id=12 bits_offset=576
> >> > >> >     'rx_queue_mapping' type_id=10 bits_offset=608
> >> > >> > [32] PTR '(anon)' type_id=31
> >> > >> > [33] STRUCT '__sk_buff' size=192 vlen=33
> >> > >> >     'len' type_id=12 bits_offset=0
> >> > >> >     'pkt_type' type_id=12 bits_offset=32
> >> > >> >     'mark' type_id=12 bits_offset=64
> >> > >> >     'queue_mapping' type_id=12 bits_offset=96
> >> > >> >     'protocol' type_id=12 bits_offset=128
> >> > >> >     'vlan_present' type_id=12 bits_offset=160
> >> > >> >     'vlan_tci' type_id=12 bits_offset=192
> >> > >> >     'vlan_proto' type_id=12 bits_offset=224
> >> > >> >     'priority' type_id=12 bits_offset=256
> >> > >> >     'ingress_ifindex' type_id=12 bits_offset=288
> >> > >> >     'ifindex' type_id=12 bits_offset=320
> >> > >> >     'tc_index' type_id=12 bits_offset=352
> >> > >> >     'cb' type_id=34 bits_offset=384
> >> > >> >     'hash' type_id=12 bits_offset=544
> >> > >> >     'tc_classid' type_id=12 bits_offset=576
> >> > >> >     'data' type_id=12 bits_offset=608
> >> > >> >     'data_end' type_id=12 bits_offset=640
> >> > >> >     'napi_id' type_id=12 bits_offset=672
> >> > >> >     'family' type_id=12 bits_offset=704
> >> > >> >     'remote_ip4' type_id=12 bits_offset=736
> >> > >> >     'local_ip4' type_id=12 bits_offset=768
> >> > >> >     'remote_ip6' type_id=28 bits_offset=800
> >> > >> >     'local_ip6' type_id=28 bits_offset=928
> >> > >> >     'remote_port' type_id=12 bits_offset=1056
> >> > >> >     'local_port' type_id=12 bits_offset=1088
> >> > >> >     'data_meta' type_id=12 bits_offset=1120
> >> > >> >     '(anon)' type_id=23 bits_offset=1152
> >> > >> >     'tstamp' type_id=15 bits_offset=1216
> >> > >> >     'wire_len' type_id=12 bits_offset=1280
> >> > >> >     'gso_segs' type_id=12 bits_offset=1312
> >> > >> >     '(anon)' type_id=30 bits_offset=1344
> >> > >> >     'gso_size' type_id=12 bits_offset=1408
> >> > >> >     'hwtstamp' type_id=15 bits_offset=1472
> >> > >> > [34] ARRAY '(anon)' type_id=12 index_type_id=16 nr_elems=5
> >> > >> > [35] CONST '(anon)' type_id=33
> >> > >> > [36] PTR '(anon)' type_id=0
> >> > >> > [37] STRUCT '(anon)' size=24 vlen=3
> >> > >> >     'type' type_id=39 bits_offset=0
> >> > >> >     'key' type_id=40 bits_offset=64
> >> > >> >     'value' type_id=41 bits_offset=128
> >> > >> > [38] ARRAY '(anon)' type_id=9 index_type_id=16 nr_elems=1
> >> > >> > [39] PTR '(anon)' type_id=38
> >> > >> > [40] PTR '(anon)' type_id=12
> >> > >> > [41] PTR '(anon)' type_id=3
> >> > >> > [42] ARRAY '(anon)' type_id=19 index_type_id=16 nr_elems=18
> >> > >> > [43] CONST '(anon)' type_id=42
> >> > >> > [44] FUNC_PROTO '(anon)' ret_type_id=36 vlen=2
> >> > >> >     '(anon)' type_id=36
> >> > >> >     '(anon)' type_id=46
> >> > >> > [45] CONST '(anon)' type_id=0
> >> > >> > [46] PTR '(anon)' type_id=45
> >> > >> > [47] FUNC_PROTO '(anon)' ret_type_id=9 vlen=1
> >> > >> >     'sk' type_id=48
> >> > >> > [48] PTR '(anon)' type_id=35
> >> > >> > [49] VAR '_license' type_id=43, linkage=static
> >> > >> > [50] VAR 'is_allow_list' type_id=5, linkage=global
> >> > >> > [51] VAR 'sd_restrictif' type_id=37, linkage=global
> >> > >> > [52] FUNC 'sd_restrictif_i' type_id=47 linkage=global
> >> > >> > [53] FUNC 'sd_restrictif_e' type_id=47 linkage=global
> >> > >> > [54] FUNC 'restrict_network_interfaces_impl' type_id=47 linkage=static
> >> > >> > [55] DATASEC '.data' size=1 vlen=1
> >> > >> >     type_id=50 offset=0 size=1 (VAR 'is_allow_list')
> >> > >> > [56] DATASEC 'license' size=18 vlen=1
> >> > >> >     type_id=49 offset=0 size=18 (VAR '_license')
> >> > >> > [57] DATASEC '.maps' size=24 vlen=1
> >> > >> >     type_id=51 offset=0 size=24 (VAR 'sd_restrictif')
> >> > >> >
> >> > >> > File before being stripped using bpftool gen object:
> >> > >> > /home/buildroot/buildroot/output/per-package/systemd/host/sbin/bpftool
> >> > >> > btf dump file
> >> > >> >
> > output/build/systemd-custom/build/src/core/bpf/restrict_ifaces/restrict-ifaces.bpf.unstripped.o
> >> > >> > format raw
> >> > >> > [1] INT 'signed char' size=1 bits_offset=0 nr_bits=8 encoding=UNKN
> >> > >> > [2] INT 'unsigned char' size=1 bits_offset=0 nr_bits=8 encoding=CHAR
> >> > >> > [3] TYPEDEF '__u8' type_id=2
> >> > >> > [4] CONST '(anon)' type_id=3
> >> > >> > [5] VOLATILE '(anon)' type_id=4
> >> > >> > [6] INT 'short int' size=2 bits_offset=0 nr_bits=16 encoding=SIGNED
> >> > >> > [7] INT 'short unsigned int' size=2 bits_offset=0 nr_bits=16 encoding=(none)
> >> > >> > [8] TYPEDEF '__u16' type_id=7
> >> > >> > [9] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
> >> > >> > [10] TYPEDEF '__s32' type_id=9
> >> > >> > [11] INT 'unsigned int' size=4 bits_offset=0 nr_bits=32 encoding=(none)
> >> > >> > [12] TYPEDEF '__u32' type_id=11
> >> > >> > [13] INT 'long long int' size=8 bits_offset=0 nr_bits=64 encoding=SIGNED
> >> > >> > [14] INT 'long long unsigned int' size=8 bits_offset=0 nr_bits=64
> >> > >> > encoding=(none)
> >> > >> > [15] TYPEDEF '__u64' type_id=14
> >> > >> > [16] INT 'long unsigned int' size=8 bits_offset=0 nr_bits=64 encoding=(none)
> >> > >> > [17] INT 'long int' size=8 bits_offset=0 nr_bits=64 encoding=SIGNED
> >> > >> > [18] INT 'char' size=1 bits_offset=0 nr_bits=8 encoding=UNKN
> >> > >> > [19] CONST '(anon)' type_id=18
> >> > >> > [20] TYPEDEF '__be16' type_id=8
> >> > >> > [21] TYPEDEF '__be32' type_id=12
> >> > >> > [22] ENUM 'bpf_map_type' encoding=UNSIGNED size=4 vlen=31
> >> > >> >     'BPF_MAP_TYPE_UNSPEC' val=0
> >> > >> >     'BPF_MAP_TYPE_HASH' val=1
> >> > >> >     'BPF_MAP_TYPE_ARRAY' val=2
> >> > >> >     'BPF_MAP_TYPE_PROG_ARRAY' val=3
> >> > >> >     'BPF_MAP_TYPE_PERF_EVENT_ARRAY' val=4
> >> > >> >     'BPF_MAP_TYPE_PERCPU_HASH' val=5
> >> > >> >     'BPF_MAP_TYPE_PERCPU_ARRAY' val=6
> >> > >> >     'BPF_MAP_TYPE_STACK_TRACE' val=7
> >> > >> >     'BPF_MAP_TYPE_CGROUP_ARRAY' val=8
> >> > >> >     'BPF_MAP_TYPE_LRU_HASH' val=9
> >> > >> >     'BPF_MAP_TYPE_LRU_PERCPU_HASH' val=10
> >> > >> >     'BPF_MAP_TYPE_LPM_TRIE' val=11
> >> > >> >     'BPF_MAP_TYPE_ARRAY_OF_MAPS' val=12
> >> > >> >     'BPF_MAP_TYPE_HASH_OF_MAPS' val=13
> >> > >> >     'BPF_MAP_TYPE_DEVMAP' val=14
> >> > >> >     'BPF_MAP_TYPE_SOCKMAP' val=15
> >> > >> >     'BPF_MAP_TYPE_CPUMAP' val=16
> >> > >> >     'BPF_MAP_TYPE_XSKMAP' val=17
> >> > >> >     'BPF_MAP_TYPE_SOCKHASH' val=18
> >> > >> >     'BPF_MAP_TYPE_CGROUP_STORAGE' val=19
> >> > >> >     'BPF_MAP_TYPE_REUSEPORT_SOCKARRAY' val=20
> >> > >> >     'BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE' val=21
> >> > >> >     'BPF_MAP_TYPE_QUEUE' val=22
> >> > >> >     'BPF_MAP_TYPE_STACK' val=23
> >> > >> >     'BPF_MAP_TYPE_SK_STORAGE' val=24
> >> > >> >     'BPF_MAP_TYPE_DEVMAP_HASH' val=25
> >> > >> >     'BPF_MAP_TYPE_STRUCT_OPS' val=26
> >> > >> >     'BPF_MAP_TYPE_RINGBUF' val=27
> >> > >> >     'BPF_MAP_TYPE_INODE_STORAGE' val=28
> >> > >> >     'BPF_MAP_TYPE_TASK_STORAGE' val=29
> >> > >> >     'BPF_MAP_TYPE_BLOOM_FILTER' val=30
> >> > >> > [23] UNION '(anon)' size=8 vlen=1
> >> > >> >     'flow_keys' type_id=29 bits_offset=0
> >> > >> > [24] STRUCT 'bpf_flow_keys' size=56 vlen=13
> >> > >> >     'nhoff' type_id=8 bits_offset=0
> >> > >> >     'thoff' type_id=8 bits_offset=16
> >> > >> >     'addr_proto' type_id=8 bits_offset=32
> >> > >> >     'is_frag' type_id=3 bits_offset=48
> >> > >> >     'is_first_frag' type_id=3 bits_offset=56
> >> > >> >     'is_encap' type_id=3 bits_offset=64
> >> > >> >     'ip_proto' type_id=3 bits_offset=72
> >> > >> >     'n_proto' type_id=20 bits_offset=80
> >> > >> >     'sport' type_id=20 bits_offset=96
> >> > >> >     'dport' type_id=20 bits_offset=112
> >> > >> >     '(anon)' type_id=25 bits_offset=128
> >> > >> >     'flags' type_id=12 bits_offset=384
> >> > >> >     'flow_label' type_id=21 bits_offset=416
> >> > >> > [25] UNION '(anon)' size=32 vlen=2
> >> > >> >     '(anon)' type_id=26 bits_offset=0
> >> > >> >     '(anon)' type_id=27 bits_offset=0
> >> > >> > [26] STRUCT '(anon)' size=8 vlen=2
> >> > >> >     'ipv4_src' type_id=21 bits_offset=0
> >> > >> >     'ipv4_dst' type_id=21 bits_offset=32
> >> > >> > [27] STRUCT '(anon)' size=32 vlen=2
> >> > >> >     'ipv6_src' type_id=28 bits_offset=0
> >> > >> >     'ipv6_dst' type_id=28 bits_offset=128
> >> > >> > [28] ARRAY '(anon)' type_id=12 index_type_id=16 nr_elems=4
> >> > >> > [29] PTR '(anon)' type_id=24
> >> > >> > [30] UNION '(anon)' size=8 vlen=1
> >> > >> >     'sk' type_id=32 bits_offset=0
> >> > >> > [31] STRUCT 'bpf_sock' size=80 vlen=14
> >> > >> >     'bound_dev_if' type_id=12 bits_offset=0
> >> > >> >     'family' type_id=12 bits_offset=32
> >> > >> >     'type' type_id=12 bits_offset=64
> >> > >> >     'protocol' type_id=12 bits_offset=96
> >> > >> >     'mark' type_id=12 bits_offset=128
> >> > >> >     'priority' type_id=12 bits_offset=160
> >> > >> >     'src_ip4' type_id=12 bits_offset=192
> >> > >> >     'src_ip6' type_id=28 bits_offset=224
> >> > >> >     'src_port' type_id=12 bits_offset=352
> >> > >> >     'dst_port' type_id=20 bits_offset=384
> >> > >> >     'dst_ip4' type_id=12 bits_offset=416
> >> > >> >     'dst_ip6' type_id=28 bits_offset=448
> >> > >> >     'state' type_id=12 bits_offset=576
> >> > >> >     'rx_queue_mapping' type_id=10 bits_offset=608
> >> > >> > [32] PTR '(anon)' type_id=31
> >> > >> > [33] STRUCT '__sk_buff' size=192 vlen=33
> >> > >> >     'len' type_id=12 bits_offset=0
> >> > >> >     'pkt_type' type_id=12 bits_offset=32
> >> > >> >     'mark' type_id=12 bits_offset=64
> >> > >> >     'queue_mapping' type_id=12 bits_offset=96
> >> > >> >     'protocol' type_id=12 bits_offset=128
> >> > >> >     'vlan_present' type_id=12 bits_offset=160
> >> > >> >     'vlan_tci' type_id=12 bits_offset=192
> >> > >> >     'vlan_proto' type_id=12 bits_offset=224
> >> > >> >     'priority' type_id=12 bits_offset=256
> >> > >> >     'ingress_ifindex' type_id=12 bits_offset=288
> >> > >> >     'ifindex' type_id=12 bits_offset=320
> >> > >> >     'tc_index' type_id=12 bits_offset=352
> >> > >> >     'cb' type_id=34 bits_offset=384
> >> > >> >     'hash' type_id=12 bits_offset=544
> >> > >> >     'tc_classid' type_id=12 bits_offset=576
> >> > >> >     'data' type_id=12 bits_offset=608
> >> > >> >     'data_end' type_id=12 bits_offset=640
> >> > >> >     'napi_id' type_id=12 bits_offset=672
> >> > >> >     'family' type_id=12 bits_offset=704
> >> > >> >     'remote_ip4' type_id=12 bits_offset=736
> >> > >> >     'local_ip4' type_id=12 bits_offset=768
> >> > >> >     'remote_ip6' type_id=28 bits_offset=800
> >> > >> >     'local_ip6' type_id=28 bits_offset=928
> >> > >> >     'remote_port' type_id=12 bits_offset=1056
> >> > >> >     'local_port' type_id=12 bits_offset=1088
> >> > >> >     'data_meta' type_id=12 bits_offset=1120
> >> > >> >     '(anon)' type_id=23 bits_offset=1152
> >> > >> >     'tstamp' type_id=15 bits_offset=1216
> >> > >> >     'wire_len' type_id=12 bits_offset=1280
> >> > >> >     'gso_segs' type_id=12 bits_offset=1312
> >> > >> >     '(anon)' type_id=30 bits_offset=1344
> >> > >> >     'gso_size' type_id=12 bits_offset=1408
> >> > >> >     'hwtstamp' type_id=15 bits_offset=1472
> >> > >> > [34] ARRAY '(anon)' type_id=12 index_type_id=16 nr_elems=5
> >> > >> > [35] CONST '(anon)' type_id=33
> >> > >> > [36] PTR '(anon)' type_id=0
> >> > >> > [37] STRUCT '(anon)' size=24 vlen=3
> >> > >> >     'type' type_id=39 bits_offset=0
> >> > >> >     'key' type_id=40 bits_offset=64
> >> > >> >     'value' type_id=41 bits_offset=128
> >> > >> > [38] ARRAY '(anon)' type_id=9 index_type_id=16 nr_elems=1
> >> > >> > [39] PTR '(anon)' type_id=38
> >> > >> > [40] PTR '(anon)' type_id=12
> >> > >> > [41] PTR '(anon)' type_id=3
> >> > >> > [42] ARRAY '(anon)' type_id=19 index_type_id=16 nr_elems=18
> >> > >> > [43] CONST '(anon)' type_id=42
> >> > >> > [44] FUNC_PROTO '(anon)' ret_type_id=36 vlen=2
> >> > >> >     '(anon)' type_id=36
> >> > >> >     '(anon)' type_id=46
> >> > >> > [45] CONST '(anon)' type_id=0
> >> > >> > [46] PTR '(anon)' type_id=45
> >> > >> > [47] FUNC_PROTO '(anon)' ret_type_id=9 vlen=1
> >> > >> >     'sk' type_id=48
> >> > >> > [48] PTR '(anon)' type_id=35
> >> > >> > [49] FUNC_PROTO '(anon)' ret_type_id=9 vlen=1
> >> > >> >     'sk' type_id=48
> >> > >> > [50] FUNC_PROTO '(anon)' ret_type_id=9 vlen=1
> >> > >> >     'sk' type_id=48
> >> > >> > [51] VAR '_license' type_id=43, linkage=static
> >> > >> > [52] VAR 'is_allow_list' type_id=5, linkage=global
> >> > >> > [53] VAR 'sd_restrictif' type_id=37, linkage=global
> >> > >> > [54] FUNC 'bpf_map_lookup_elem' type_id=44 linkage=global
> >> > >> > [55] FUNC 'sd_restrictif_i' type_id=47 linkage=global
> >> > >> > [56] FUNC 'sd_restrictif_e' type_id=49 linkage=global
> >> > >> > [57] FUNC 'restrict_network_interfaces_impl' type_id=50 linkage=static
> >> > >> > [58] DATASEC 'license' size=0 vlen=1
> >> > >> >     type_id=51 offset=0 size=18 (VAR '_license')
> >> > >> > [59] DATASEC '.maps' size=0 vlen=1
> >> > >> >     type_id=53 offset=0 size=24 (VAR 'sd_restrictif')
> >> > >> > [60] DATASEC '.data' size=0 vlen=1
> >> > >> >     type_id=52 offset=0 size=1 (VAR 'is_allow_list')
> >> > >> >
> >> > >> >>
> >> > >> >> >> GCC is wrong, clearly. This function is global ([0]) and libbpf
> >> > >> >> >> expects it to be marked as such in BTF.
> >> > >> >> >>
> >> > >> >> >>
> >> > >> >
> > https://github.com/systemd/systemd/blob/main/src/core/bpf/restrict_ifaces/restrict-ifaces.bpf.c#L42-L50
> >> > >> >> >>
> >> > >> >> >>
> >> > >> >> >>> GCC:
> >> > >> >> >>>
> >> > >> >> >>> [1] INT 'signed char' size=1 bits_offset=0 nr_bits=8 encoding=UNKN
> >> > >> >> >>> [2] INT 'unsigned char' size=1 bits_offset=0 nr_bits=8 encoding=CHAR
> >> > >> >> >>> [3] TYPEDEF '__u8' type_id=2
> >> > >> >> >>> [4] CONST '(anon)' type_id=3
> >> > >> >> >>
> >> > >> >> >> [...]



[Index of Archives]     [Linux Samsung SoC]     [Linux Rockchip SoC]     [Linux Actions SoC]     [Linux for Synopsys ARC Processors]     [Linux NFS]     [Linux NILFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]


  Powered by Linux