On Mon, Feb 27, 2023 at 10:05 AM KP Singh <kpsingh@xxxxxxxxxx> wrote: > > On Mon, Feb 27, 2023 at 6:32 PM Andrii Nakryiko > <andrii.nakryiko@xxxxxxxxx> wrote: > > > > On Mon, Feb 27, 2023 at 6:17 AM Eduard Zingerman <eddyz87@xxxxxxxxx> wrote: > > > > > > On Sun, 2023-02-26 at 03:03 +0200, Eduard Zingerman wrote: > > > > On Sat, 2023-02-25 at 20:50 +0000, Matt Bobrowski wrote: > > > > > Sorry Eduard, I replied late last night although the email bounced due > > > > > to exceeding the mail char limit. Let's try attaching a compressed > > > > > variant of the requested files, which includes the compiled kernel's > > > > > BTF and the kernel's config. > > > > > > > > Hi Matt, > > > > > > > > I tried using your config but still can't reproduce the issue. > > > > Will try to do it using debian 12 chroot tomorrow or on Monday. > > > > > > Hi Matt, > > > > > > Short update: > > > I've reproduced the issue with multiple STRUCT 'linux_binprm' BTF IDs > > > in Debian testing chroot, thank you for providing all details. > > > Attaching the instructions in the end of the email. > > > Need some time to analyze pahole behavior. > > > > > > > Try using [0] to pinpoint what actually is different between any two > > linux_binprm definitions. I've hacked up this "tool" last time I had > > to pinpoint where two BTF types diverge, maybe it will save you a bit > > of time as well. I'd like to put this functionality into btfdump > > ([1]), but I didn't get to it yet, unfortunately. > > > > It's not just linux_binprm but a bunch of other structs (quite a lot > of them that seem to diverge) yes, task_struct is usually a thing that suffers from such duplications, as it forms a huge graph of types. Which is where that btfdiff "tool" comes handly, it recursively compares side-by-side two types that are supposed to be equal (but are not), by ignoring BTF type IDs and trying to pin point actual differences (like STRUCT vs FWD, or extra field, or whatever). It just needs two starting type IDs. > > [...] > WARN: multiple IDs found for 'sock_common': 4400, 53212 - using 4400 > WARN: multiple IDs found for 'request_sock': 4458, 53257 - using 4458 > WARN: multiple IDs found for 'task_struct': 265, 55830 - using 265 > WARN: multiple IDs found for 'file': 474, 55854 - using 474 > WARN: multiple IDs found for 'vm_area_struct': 480, 55857 - using 480 > WARN: multiple IDs found for 'seq_file': 670, 55891 - using 670 [...] > > I was able to "fix" this with using clang 16.x and the following hacky pahole: > > 030e3b4 - core: Add DW_TAG_unspecified_type to tag__is_tag_type() set > (HEAD) (2023-02-26 Arnaldo Carvalho de Melo) > f20515b - dwarves: support DW_TAG_atomic_type (2023-02-26 David Lamparter) > de24234 - pahole: Prep 1.24 (tag: v1.24) (2022-08-22 Arnaldo Carvalho de Melo) > d6c9528 - dwarf_loader: Encode char type as signed (2022-08-10 Yonghong Song) > > and the the following patch on top: > I'd start with understanding what BTF and DWARF differences are causing the issue before trying to come up with the fix. For that we don't even need config or repro steps, it should be enough to share vmlinux with BTF and DWARF, and start from there. But I'm sure Eduard is on top of this already (especially that he can repro the issue now).