On Fri, Jun 21, 2024 at 6:52 AM Brian Norris <briannorris@xxxxxxxxxxxx> wrote: > > On Wed, Jun 19, 2024 at 03:21:42PM +0900, Masahiro Yamada wrote: > > Kbuild provides scripts/Makefile.host to build host programs used for > > building the kernel. Unfortunately, there are two exceptions that opt > > out of Kbuild. The build system under tools/ is a cheesy replica, and > > is always a disaster. I was recently poked about a problem in the tools > > build issue, which I do not maintain (and nobody maintains). [1] > > (Side note: I hope I haven't placed undue burden on you; I understood > you don't maintain tools/ and that it didn't use Kbuild. I only "poked" > you because the original bug report I was replying to had you and > linux-kbuild on CC already. And I appreciate your engagement, even if > the bugs are due to intentional forking.) I did not mean to express my complaint particularly with the previous thread. It is not the first time that the tools/ build issue arose. I will drop the references to the threads. > But anyway, I agree that clearer documentation and recommendations could > be helpful here. To that end, some dumb questions below, as I'm not sure > if this fully serves its purpose as-is: > > > Without a comment, somebody might believe this is the right location > > because that is where objtool lives, even when a more robust Kbuild > > syntax satisfies their needs. [2] > > > > [1]: https://lore.kernel.org/linux-kbuild/ZnIYWBgrJ-IJtqK8@xxxxxxxxxx/T/#m8ece130dd0e23c6f2395ed89070161948dee8457 > > [2]: https://lore.kernel.org/all/20240618200501.GA1611012@xxxxxxxxxx/ > > > > Signed-off-by: Masahiro Yamada <masahiroy@xxxxxxxxxx> > > --- > > > > Makefile | 5 +++++ > > 1 file changed, 5 insertions(+) > > > > diff --git a/Makefile b/Makefile > > index 471f2df86422..ba070596ad4e 100644 > > --- a/Makefile > > +++ b/Makefile > > @@ -1331,6 +1331,11 @@ prepare: tools/bpf/resolve_btfids > > endif > > endif > > > > +# README > > +# The tools build system is not a part of Kbuild. Before adding yet another > > +# tools/* here, please consider if the standard "hostprogs" syntax satisfies > > +# your needs. > > + > > Some clarifying questions / statements-as-questions: > > * nothing in tools/ uses Kbuild, right? (even stuff that uses KBUILD_* > names is just an imitative port, right?) Correct. You can build a tool from multiple directory locations. For example, you can compile the 'perf' in multiple locations. [1] From the top of the kernel tree $ make tools/perf [2] From the tools/ directory $ cd tools $ make perf [3] From the tools/perf/ directory $ cd tools/perf $ make When you do [2] or [3], the top-level Makefile is not parsed. If necessary, the tools build system copies code from Kbuild. > * not everything in tools/ is actually promoted to a high-level target, > that affects this top-level Makefile. Are you only concerned about > stuff that pretends to be integrated in the top-level kernel Makefile? > (If not, then it seems like placing the README comments only in this > Makefile is a poor choice.) The tool build is integrated as a pattern rule in the top Makefile. (tools/%) So, you can build other tools from the top Makefile. See commit ea01fa9f63aef, which did not get Ack from any Kbuild maintainer, and caused subsequent troubles, and the benefit of which I still do not understand. Supporting "make tools/perf" in addition to "make -C tools perf" only saved a few characters to type. So, the problem remains, unless I revert ea01fa9f63aef. I decided to not care about it too much, as long as such tools are not used during the kernel build. I am really worried about objtool and resolve_btfids, as these two are used for building the kernel. > * is the "standard hostprogs" recommendation a general recommendation, > for all sorts of kept-in-the-kernel-tree host tools? Is the > recommendation to "use Kbuild" or to "avoid putting your tool in > tools/*"? Is it possible (recommended?) to plumb Kbuild stuff into > tools/, even if other parts won't migrate? I do not know. They are different build systems with different designs. Kbuild always works in the top of the output directory. Kbuild changes the working directory at most once if O= is given, but otherwise, it never changes the working directory during the build. The tools/ build system changes the working directory every time it invokes a new Make, and compiles the tool in its source directory. I do not know if all tools want to Kbuild. (the same applied to kselftest) I think I can convert objtool and resolve_btfids to the Kbuild way. > > As is, I can't tell if this is telling people to avoid adding new stuff > to tools/ entirely, or just to only add to tools/ if you're able to > remain completely isolated from the rest of the kernel build -- as soon > as you want to play some part in the Kbuild-covered part of the tree, > you need to use Kbuild. See the code in the top Makefile. 'prepare' depends on tools/objtool and tools/bpf/resolve_btfids. If other tools are not prerequisites of 'scripts', Kbuild will not compile them. > > If I'm inferring the right answers to the above, then maybe an improved > wording could be something like: > > "The tools build system is not a part of Kbuild and tends to introduce > its own unique issues. If you need to integrate a new tool into Kbuild, > please consider locating that tool outside the tools/ tree and using the > standard Kbuild "hostprogs" syntax instead of adding a new tools/* entry > here." I am fine with this description. Nicolas suggested a link to Documentation/kbuild/makefiles.rst We can combine the two. # The tools build system is not a part of Kbuild and tends to introduce # its own unique issues. If you need to integrate a new tool into Kbuild, # please consider locating that tool outside the tools/ tree and using the # standard Kbuild "hostprogs" syntax instead of adding a new tools/* entry # here. See Documentation/kbuild/makefiles.rst for details. > It's possible I'm playing mental acrobatics here in my reading too. > > Either way, I think this is a good trajectory: > > Reviewed-by: Brian Norris <briannorris@xxxxxxxxxxxx> > > Regards, > Brian > > > PHONY += resolve_btfids_clean > > > > resolve_btfids_O = $(abspath $(objtree))/tools/bpf/resolve_btfids > > -- > > 2.43.0 > > -- Best Regards Masahiro Yamada
