On Wed, Oct 27, 2021 at 11:23:55PM IST, Andrii Nakryiko wrote: > On Wed, Oct 27, 2021 at 1:53 AM Jiri Olsa <jolsa@xxxxxxxxxx> wrote: > > > > On Tue, Oct 26, 2021 at 09:12:31PM -0700, Andrii Nakryiko wrote: > > > On Tue, Oct 26, 2021 at 5:03 AM Jiri Olsa <jolsa@xxxxxxxxxx> wrote: > > > > > > > > On Mon, Oct 25, 2021 at 09:54:48PM -0700, Andrii Nakryiko wrote: > > > > > On Sat, Oct 23, 2021 at 5:05 AM Jiri Olsa <jolsa@xxxxxxxxxx> wrote: > > > > > > > > > > > > hi, > > > > > > I'm trying to enable BTF for kernel module in fedora, > > > > > > and I'm getting big increase on modules sizes on s390x arch. > > > > > > > > > > > > Size of modules in total - kernel dir under /lib/modules/VER/ > > > > > > from kernel-core and kernel-module packages: > > > > > > > > > > > > current new > > > > > > aarch64 60M 76M > > > > > > ppc64le 53M 66M > > > > > > s390x 21M 41M > > > > > > x86_64 64M 79M > > > > > > > > > > > > The reason for higher increase on s390x was that dedup algorithm > > > > > > did not detect some of the big kernel structs like 'struct module', > > > > > > so they are duplicated in the kernel module BTF data. The s390x > > > > > > has many small modules that increased significantly in size because > > > > > > of that even after compression. > > > > > > > > > > > > First issues was that the '--btf_gen_floats' option is not passed > > > > > > to pahole for kernel module BTF generation. > > > > > > > > > > > > The other problem is more tricky and is the reason why this patchset > > > > > > is RFC ;-) > > > > > > > > > > > > The s390x compiler generates multiple definitions of the same struct > > > > > > and dedup algorithm does not seem to handle this at the moment. > > > > > > > > > > > > I put the debuginfo and btf dump of the s390x pnet.ko module in here: > > > > > > http://people.redhat.com/~jolsa/kmodbtf/ > > > > > > > > > > > > Please let me know if you'd like to see other info/files. > > > > > > > > > > > > > > > > Hard to tell what's going on without vmlinux itself. Can you upload a > > > > > corresponding kernel image with BTF in it? > > > > > > > > sure, uploaded > > > > > > > > > > vmlinux.btfdump: > > > > > > [174] FLOAT 'float' size=4 > > > [175] FLOAT 'double' size=8 > > > > > > VS > > > > > > pnet.btfdump: > > > > > > [89318] INT 'float' size=4 bits_offset=0 nr_bits=32 encoding=(none) > > > [89319] INT 'double' size=8 bits_offset=0 nr_bits=64 encoding=(none) > > > > ugh, that's with no fix applied, sry > > > > I applied the first patch and uploaded new files > > > > now when I compare the 'module' struct from vmlinux: > > > > [885] STRUCT 'module' size=1280 vlen=70 > > > > and same one from pnet.ko: > > > > [89323] STRUCT 'module' size=1280 vlen=70 > > > > they seem to completely match, all the fields > > and yet it still appears in the kmod's BTF > > > > Ok, now struct module is identical down to the types referenced from > the fields, which means it should have been deduplicated completely. > This will require a more time-consuming debugging, though, so I'll put > it on my TODO list for now. If you get to this earlier, see where the > equivalence check fails in btf_dedup (sprinkle debug outputs around to > see what's going on). > Hello Andrii, I think I'm seeing something similar when working on the conntrack patches [0], I was looking to match whether the type in a PTR_TO_BTF_ID register is same as struct nf_conn, but it seems that there can be two BTF IDs for the same struct type. When doing bpftool dump, I see: ; bpftool btf dump file /sys/kernel/btf/vmlinux format raw | grep nf_conn ... [89224] STRUCT 'nf_conn' size=256 vlen=15 ... ; bpftool btf dump file /sys/kernel/btf/nf_conntrack format raw | grep nf_conn ... [103077] STRUCT 'nf_conn' size=256 vlen=15 [104988] STRUCT 'nf_conn' size=256 vlen=15 [106490] STRUCT 'nf_conn' size=256 vlen=15 [108187] STRUCT 'nf_conn' size=256 vlen=15 ... Inside the kernel, when trying to match both, register PTR_TO_BTF_ID refers to the nf_conntrack BTF ID, while the BTF_ID_LIST resolves to the one in vmlinux, this ends up making the job of matching the two struct types a bit difficult (for now, I am thinking of going with btf_struct_ids_match). My original plan was to compare the result of btf_types_by_id. [0]: https://github.com/kkdwivedi/linux/commits/conntrack-bpf > > thanks, > > jirka > > > > > > > > > > > > jirka > > > > > > > > > > > > > > > I found code in dedup that seems to handle such situation for arrays, > > > > > > and added 'some' fix for structs. With that change I can no longer > > > > > > see vmlinux's structs in kernel module BTF data, but I have no idea > > > > > > if that breaks anything else. > > > > > > > > > > > > thoughts? thanks, > > > > > > jirka > > > > > > > > > > > > > > > > > > --- > > > > > > Jiri Olsa (2): > > > > > > kbuild: Unify options for BTF generation for vmlinux and modules > > > > > > bpf: Add support to detect and dedup instances of same structs > > > > > > > > > > > > Makefile | 3 +++ > > > > > > scripts/Makefile.modfinal | 2 +- > > > > > > scripts/link-vmlinux.sh | 11 +---------- > > > > > > scripts/pahole-flags.sh | 20 ++++++++++++++++++++ > > > > > > tools/lib/bpf/btf.c | 12 ++++++++++-- > > > > > > 5 files changed, 35 insertions(+), 13 deletions(-) > > > > > > create mode 100755 scripts/pahole-flags.sh > > > > > > > > > > > > > > > > > > > > -- Kartikeya
