On Thu, Jun 27, 2024 at 04:02:46AM +0900, Masahiro Yamada wrote: > On Fri, Jun 21, 2024 at 6:52 AM Brian Norris <briannorris@xxxxxxxxxxxx> wrote: > > > > On Wed, Jun 19, 2024 at 03:21:42PM +0900, Masahiro Yamada wrote: > > > Kbuild provides scripts/Makefile.host to build host programs used for > > > building the kernel. Unfortunately, there are two exceptions that opt > > > out of Kbuild. The build system under tools/ is a cheesy replica, and > > > is always a disaster. I was recently poked about a problem in the tools > > > build issue, which I do not maintain (and nobody maintains). [1] > > > > (Side note: I hope I haven't placed undue burden on you; I understood > > you don't maintain tools/ and that it didn't use Kbuild. I only "poked" > > you because the original bug report I was replying to had you and > > linux-kbuild on CC already. And I appreciate your engagement, even if > > the bugs are due to intentional forking.) > > > I did not mean to express my complaint particularly with the previous thread. > > It is not the first time that the tools/ build issue arose. > > > I will drop the references to the threads. > > > > > But anyway, I agree that clearer documentation and recommendations could > > be helpful here. To that end, some dumb questions below, as I'm not sure > > if this fully serves its purpose as-is: > > > > > Without a comment, somebody might believe this is the right location > > > because that is where objtool lives, even when a more robust Kbuild > > > syntax satisfies their needs. [2] > > > > > > [1]: https://lore.kernel.org/linux-kbuild/ZnIYWBgrJ-IJtqK8@xxxxxxxxxx/T/#m8ece130dd0e23c6f2395ed89070161948dee8457 > > > [2]: https://lore.kernel.org/all/20240618200501.GA1611012@xxxxxxxxxx/ > > > > > > Signed-off-by: Masahiro Yamada <masahiroy@xxxxxxxxxx> > > > --- > > > > > > Makefile | 5 +++++ > > > 1 file changed, 5 insertions(+) > > > > > > diff --git a/Makefile b/Makefile > > > index 471f2df86422..ba070596ad4e 100644 > > > --- a/Makefile > > > +++ b/Makefile > > > @@ -1331,6 +1331,11 @@ prepare: tools/bpf/resolve_btfids > > > endif > > > endif > > > > > > +# README > > > +# The tools build system is not a part of Kbuild. Before adding yet another > > > +# tools/* here, please consider if the standard "hostprogs" syntax satisfies > > > +# your needs. > > > + > > > > Some clarifying questions / statements-as-questions: > > > > * nothing in tools/ uses Kbuild, right? (even stuff that uses KBUILD_* > > names is just an imitative port, right?) > > > Correct. > > You can build a tool from multiple directory locations. > > For example, you can compile the 'perf' in multiple locations. > > > [1] From the top of the kernel tree > > $ make tools/perf > > > [2] From the tools/ directory > > $ cd tools > $ make perf > > > [3] From the tools/perf/ directory > > $ cd tools/perf > $ make > > > > When you do [2] or [3], the top-level Makefile is not parsed. > > If necessary, the tools build system copies code from Kbuild. > > > > > > * not everything in tools/ is actually promoted to a high-level target, > > that affects this top-level Makefile. Are you only concerned about > > stuff that pretends to be integrated in the top-level kernel Makefile? > > (If not, then it seems like placing the README comments only in this > > Makefile is a poor choice.) > > > The tool build is integrated as a pattern rule in the top Makefile. > (tools/%) > > > So, you can build other tools from the top Makefile. > > > See commit ea01fa9f63aef, which did not get Ack from any Kbuild > maintainer, and caused subsequent troubles, and the benefit > of which I still do not understand. > > > Supporting "make tools/perf" in addition to "make -C tools perf" > only saved a few characters to type. > > > So, the problem remains, unless I revert ea01fa9f63aef. > > I decided to not care about it too much, as long as > such tools are not used during the kernel build. > > I am really worried about objtool and resolve_btfids, > as these two are used for building the kernel. > > > > > > > > * is the "standard hostprogs" recommendation a general recommendation, > > for all sorts of kept-in-the-kernel-tree host tools? Is the > > recommendation to "use Kbuild" or to "avoid putting your tool in > > tools/*"? Is it possible (recommended?) to plumb Kbuild stuff into > > tools/, even if other parts won't migrate? > > > I do not know. > > They are different build systems with different designs. > > Kbuild always works in the top of the output directory. > Kbuild changes the working directory at most once if O= is given, > but otherwise, it never changes the working directory during the build. > > > The tools/ build system changes the working directory every time > it invokes a new Make, and compiles the tool in its source directory. > > > I do not know if all tools want to Kbuild. > (the same applied to kselftest) > > I think I can convert objtool and resolve_btfids to the Kbuild way. > > > > > > As is, I can't tell if this is telling people to avoid adding new stuff > > to tools/ entirely, or just to only add to tools/ if you're able to > > remain completely isolated from the rest of the kernel build -- as soon > > as you want to play some part in the Kbuild-covered part of the tree, > > you need to use Kbuild. > > > See the code in the top Makefile. > > 'prepare' depends on tools/objtool and tools/bpf/resolve_btfids. > > If other tools are not prerequisites of 'scripts', > Kbuild will not compile them. > > > > > > > > If I'm inferring the right answers to the above, then maybe an improved > > wording could be something like: > > > > "The tools build system is not a part of Kbuild and tends to introduce > > its own unique issues. If you need to integrate a new tool into Kbuild, > > please consider locating that tool outside the tools/ tree and using the > > standard Kbuild "hostprogs" syntax instead of adding a new tools/* entry > > here." > > > > I am fine with this description. > > > Nicolas suggested a link to Documentation/kbuild/makefiles.rst > > We can combine the two. > > > # The tools build system is not a part of Kbuild and tends to introduce > # its own unique issues. If you need to integrate a new tool into Kbuild, > # please consider locating that tool outside the tools/ tree and using the > # standard Kbuild "hostprogs" syntax instead of adding a new tools/* entry > # here. See Documentation/kbuild/makefiles.rst for details. yeah, thanks. Sounds good to me, too. Kind regards, Nicolas
