On Fri, May 24, 2024 at 2:54 AM Brian Norris <briannorris@xxxxxxxxxxxx> wrote:
>
> Necromancing an old thread, since it identifies the same problem I've
> been poking at recently:
>
> On Thu, May 18, 2023 at 05:26:28PM +0900, Masahiro Yamada wrote:
> > (+CC: Josh Poimboeuf,Peter Zijlstra, objtool maintainer)
> >
> > On Thu, May 18, 2023 at 8:27 AM Bjorn Helgaas <helgaas@xxxxxxxxxx> wrote:
> > >
> > > This is on v6.4-rc1. I fat-fingered the make target (I intended
> > > "pciehp.o", not "pciehp.c"), then interrupted the build when I noticed
> > > my mistake:
> > >
> > > 06:04:15 ~/linux (hotplug)$ make drivers/pci/hotplug/pciehp.c
> > > SYNC include/config/auto.conf.cmd
> > > ^Cmake: *** [include/config/auto.conf.cmd] Deleting file 'include/generated/rustc_cfg'
> > > make: *** [include/config/auto.conf.cmd] Deleting file 'include/generated/autoconf.h'
> > > make[2]: *** [scripts/kconfig/Makefile:77: syncconfig] Interrupt
> > > make[1]: *** [Makefile:692: syncconfig] Interrupt
> > > make: *** [Makefile:793: include/config/auto.conf.cmd] Interrupt
> > >
> > > Subsequent builds now fail ("pciehp.o" is *also* an incorrect target,
> > > but doesn't seem related to the error):
> > >
> > > 06:04:22 ~/linux (hotplug)$ make drivers/pci/hotplug/pciehp.o
> > > SYNC include/config/auto.conf.cmd
> > > UPD include/config/kernel.release
> > > UPD include/generated/utsrelease.h
> > > UPD include/generated/compile.h
> > > CC scripts/mod/empty.o
> > > MKELF scripts/mod/elfconfig.h
> > > HOSTCC scripts/mod/modpost.o
> > > CC scripts/mod/devicetable-offsets.s
> > > HOSTCC scripts/mod/file2alias.o
> > > HOSTCC scripts/mod/sumversion.o
> > > HOSTLD scripts/mod/modpost
> > > CC kernel/bounds.s
> > > CC arch/x86/kernel/asm-offsets.s
> > > CALL scripts/checksyscalls.sh
> > > DESCEND objtool
> > > HOSTCC /home/bjorn/linux/tools/objtool/fixdep.o
> > > HOSTLD /home/bjorn/linux/tools/objtool/fixdep-in.o
> > > LINK /home/bjorn/linux/tools/objtool/fixdep
> > > make[4]: *** No rule to make target '/usr/include/x86_64-linux-gnu/bits/sys_errlist.h', needed by '/home/bjorn/linux/tools/objtool/libsubcmd/exec-cmd.o'. Stop.
> > > make[3]: *** [Makefile:80: /home/bjorn/linux/tools/objtool/libsubcmd/libsubcmd-in.o] Error 2
> > > make[2]: *** [Makefile:78: /home/bjorn/linux/tools/objtool/libsubcmd/libsubcmd.a] Error 2
> > > make[1]: *** [Makefile:73: objtool] Error 2
> > > make: *** [Makefile:1440: tools/objtool] Error 2
> > >
> > > I finally got the right target, but the build still fails:
> > >
> > > 06:04:39 ~/linux (hotplug)$ make drivers/pci/hotplug/
> > > CALL scripts/checksyscalls.sh
> > > DESCEND objtool
> > > make[4]: *** No rule to make target '/usr/include/x86_64-linux-gnu/bits/sys_errlist.h', needed by '/home/bjorn/linux/tools/objtool/libsubcmd/exec-cmd.o'. Stop.
> > > make[3]: *** [Makefile:80: /home/bjorn/linux/tools/objtool/libsubcmd/libsubcmd-in.o] Error 2
> > > make[2]: *** [Makefile:78: /home/bjorn/linux/tools/objtool/libsubcmd/libsubcmd.a] Error 2
> > > make[1]: *** [Makefile:73: objtool] Error 2
> > > make: *** [Makefile:1440: tools/objtool] Error 2
> > >
> > > After "make distclean", everything works as expected, so maybe this is
> > > just the expected behavior after my initial user error? I dunno; it
> > > seemed surprising. Just FYI.
>
> I believe we've been hitting a similar issue at $JOB, which looks like
> the following (this is on a 5.15-ish kernel, but AFAICT everything is
> still relevant):
>
> make[5]: *** No rule to make target '[...]/tools/include/linux/compiler.h', needed by '[...]/tools/bpf/resolve_btfids/libsubcmd/exec-cmd.o'. Stop.
> make[4]: *** [Makefile:59: [...]/tools/bpf/resolve_btfids/libsubcmd/libsubcmd-in.o] Error 2
> make[3]: *** [Makefile:45: [...]/tools/bpf/resolve_btfids//libsubcmd/libsubcmd.a] Error 2
> make[2]: *** [Makefile:72: bpf/resolve_btfids] Error 2
> make[1]: *** [[...]/Makefile:1401: tools/bpf/resolve_btfids] Error 2
>
> This particular case happens for us when the source tree is moving, but
> we're sharing an O= cache. This may or may not be a good idea, but
> AFAICT there still is a real bug underneath, which I explore below.
>
> > I do not know what is happening on your build machine,
> > but judging from the error log, something went wrong
> > while building objtool.
> >
> > objtool Makefile is not a part of Kbuild.
> > The maintainers of objtool may have some insight.
>
> I'm no maintainer, but I found that the .exec-cmd.o.cmd dep file is
> generated incorrectly due to improper fixdep dependencies:
>
> $ head -2 [...]/tools/bpf/resolve_btfids/libsubcmd/.exec-cmd.o.cmd
> # cannot find fixdep ([...]/tools/bpf/resolve_btfids/libsubcmd//fixdep)
> # using basic dep data
>
> Now, this soft error is normally OK, as long as you don't have any
> missing or moved headers. But if these moved around, then normally the
> fixdep'd dependencies would help rebuild (and regenerate .cmd files with
> new paths) silently.
>
> The bad .cmd file is reliably reproduced by:
>
> # for an easier-to-build target that also builds libsubcmd:
> cd tools/objtool
> # for maximum cleanliness:
> git clean -xfd ..
> make
> head -2 libsubcmd/.exec-cmd.o.cmd
>
> (NB: if you look hard enough, you'll notice that we have a similar
> "cannot find fixdep" error for tools/.../.fixdep.o.cmd too. I have some
> analysis at https://issuetracker.google.com/313508829#comment32 --
> this one is publicly accessible -- but its solution would be more
> complex. I may raise a separate thread.)
>
> The following patch fixes libsubcmd stuff for me. I can resubmit in a
> proper patch form if desired, or feel free to scrape it as-is.
>
> Signed-off-by: Brian Norris <briannorris@xxxxxxxxxxxx>
> ---
>
> diff --git a/tools/lib/subcmd/Makefile b/tools/lib/subcmd/Makefile
> index b87213263a5e..59b09f280e49 100644
> --- a/tools/lib/subcmd/Makefile
> +++ b/tools/lib/subcmd/Makefile
> @@ -76,7 +76,7 @@ include $(srctree)/tools/build/Makefile.include
>
> all: fixdep $(LIBFILE)
>
> -$(SUBCMD_IN): FORCE
> +$(SUBCMD_IN): fixdep FORCE
> @$(MAKE) $(build)=libsubcmd
>
> $(LIBFILE): $(SUBCMD_IN)
I may not fully understand the design policy of the tools/ build system,
but this fix is presumably correct because the 'fixdep' binary
is needed in each sub-directory for it to work correctly.
tools/bpf/resolve_btfids/libsubcmd/.exec-cmd.o.cmd must
be generated by tools/bpf/resolve_btfids/libsubcmd/fixdep
instead of by tools/bpf/resolve_btfids/fixdep.
But, fixing tools/lib/subcmd/Makefile is not enough.
*.cmd files under tools/bpf/resolve_btfids/libbpf/staticobjs/
are broken for the same reason.
So, this is fundamentally broken in many places.
And, as you noted, there is no easy way to fix .fixdep.o.cmd
Your description in
https://issuetracker.google.com/issues/313508829#comment32
is all correct.
"can we just use Kbuild?" is a good question.
I do not understand why they use fragile build systems,
where obviously they cannot do it correctly.
In fact, I submitted a patch to migrate objtool to Kbuild
because that fixes all the issues cleanly.
The objtool maintainers rejected it.
https://lore.kernel.org/linux-kbuild/1551764896-8453-3-git-send-email-yamada.masahiro@xxxxxxxxxxxxx/
Not only the build system.
He also refused to participate in the standard Documentation
directory.
tools/objtool/Documentation/objtool.txt still resides in its own directory.
--
Best Regards
Masahiro Yamada
