On Mon, Feb 27, 2023 at 11:29 AM Eduard Zingerman <eddyz87@xxxxxxxxx> wrote: > > On Mon, 2023-02-27 at 11:24 -0800, Andrii Nakryiko wrote: > > On Mon, Feb 27, 2023 at 10:05 AM KP Singh <kpsingh@xxxxxxxxxx> wrote: > > > > > > On Mon, Feb 27, 2023 at 6:32 PM Andrii Nakryiko > > > <andrii.nakryiko@xxxxxxxxx> wrote: > > > > > > > > On Mon, Feb 27, 2023 at 6:17 AM Eduard Zingerman <eddyz87@xxxxxxxxx> wrote: > > > > > > > > > > On Sun, 2023-02-26 at 03:03 +0200, Eduard Zingerman wrote: > > > > > > On Sat, 2023-02-25 at 20:50 +0000, Matt Bobrowski wrote: > > > > > > > Sorry Eduard, I replied late last night although the email bounced due > > > > > > > to exceeding the mail char limit. Let's try attaching a compressed > > > > > > > variant of the requested files, which includes the compiled kernel's > > > > > > > BTF and the kernel's config. > > > > > > > > > > > > Hi Matt, > > > > > > > > > > > > I tried using your config but still can't reproduce the issue. > > > > > > Will try to do it using debian 12 chroot tomorrow or on Monday. > > > > > > > > > > Hi Matt, > > > > > > > > > > Short update: > > > > > I've reproduced the issue with multiple STRUCT 'linux_binprm' BTF IDs > > > > > in Debian testing chroot, thank you for providing all details. > > > > > Attaching the instructions in the end of the email. > > > > > Need some time to analyze pahole behavior. > > > > > > > > > > > > > Try using [0] to pinpoint what actually is different between any two > > > > linux_binprm definitions. I've hacked up this "tool" last time I had > > > > to pinpoint where two BTF types diverge, maybe it will save you a bit > > > > of time as well. I'd like to put this functionality into btfdump > > > > ([1]), but I didn't get to it yet, unfortunately. > > > > > > > > > > It's not just linux_binprm but a bunch of other structs (quite a lot > > > of them that seem to diverge) > > > > yes, task_struct is usually a thing that suffers from such > > duplications, as it forms a huge graph of types. Which is where that > > btfdiff "tool" comes handly, it recursively compares side-by-side two > > types that are supposed to be equal (but are not), by ignoring BTF > > type IDs and trying to pin point actual differences (like STRUCT vs > > FWD, or extra field, or whatever). It just needs two starting type > > IDs. > > Thank you for links to these tools! > > > > > > > > > [...] > > > WARN: multiple IDs found for 'sock_common': 4400, 53212 - using 4400 > > > WARN: multiple IDs found for 'request_sock': 4458, 53257 - using 4458 > > > WARN: multiple IDs found for 'task_struct': 265, 55830 - using 265 > > > WARN: multiple IDs found for 'file': 474, 55854 - using 474 > > > WARN: multiple IDs found for 'vm_area_struct': 480, 55857 - using 480 > > > WARN: multiple IDs found for 'seq_file': 670, 55891 - using 670 > > > > > > [...] > > > > > > > > I was able to "fix" this with using clang 16.x and the following hacky pahole: > > > > > > 030e3b4 - core: Add DW_TAG_unspecified_type to tag__is_tag_type() set > > > (HEAD) (2023-02-26 Arnaldo Carvalho de Melo) > > > f20515b - dwarves: support DW_TAG_atomic_type (2023-02-26 David Lamparter) > > > de24234 - pahole: Prep 1.24 (tag: v1.24) (2022-08-22 Arnaldo Carvalho de Melo) > > > d6c9528 - dwarf_loader: Encode char type as signed (2022-08-10 Yonghong Song) > > > > > > and the the following patch on top: > > > > > > > I'd start with understanding what BTF and DWARF differences are > > causing the issue before trying to come up with the fix. For that we > > don't even need config or repro steps, it should be enough to share > > vmlinux with BTF and DWARF, and start from there. > > > > Yes, I suspect that there is some kind of unanticipated > anomaly for some DWARF encoding for some kind of objects, > just need to find the root for the diverging type hierarchies. > > > But I'm sure Eduard is on top of this already (especially that he can > > repro the issue now). > > I'm working on it, nothing to report yet, but I'm working on it. > Thanks, please keep us posted!
