On Tue, 9 Mar 2021, Rasmus Villemoes wrote: > On 09/03/2021 20.54, Nicolas Pitre wrote: > > On Wed, 10 Mar 2021, Masahiro Yamada wrote: > > > > >>> I'm not sure I do understand every detail here, especially since it is > >>> so far away from the version that I originally contributed. But the > >>> concept looks good. > >>> > >>> I still think that there is no way around a recursive approach to get > >>> the maximum effect with LTO, but given that true LTO still isn't applied > >>> to mainline after all those years, the recursive approach brings > >>> nothing. Maybe that could be revisited if true LTO ever makes it into > >>> mainline, and the desire to reduce the binary size is still relevant > >>> enough to justify it. > >> > >> Hmm, I am confused. > >> > >> Does this patch change the behavior in the > >> combination with the "true LTO"? > >> > >> Please let me borrow this sentence from your article: > >> "But what LTO does is more like getting rid of branches that simply > >> float in the air without being connected to anything or which have > >> become loose due to optimization." > >> (https://lwn.net/Articles/746780/) > >> > >> This patch throws unneeded EXPORT_SYMBOL metadata > >> into the /DISCARD/ section of the linker script. > >> > >> The approach is different (preprocessor vs linker), but > >> we will still get the same result; the unneeded > >> EXPORT_SYMBOLs are disconnected from the main trunk. > >> > >> Then, the true LTO will remove branches floating in the air, > >> right? > >> > >> So, what will be lost by this patch? > > > > Let's say you have this in module_foo: > > > > int foo(int x) > > { > > return 2 + bar(x); > > } > > EXPORT_SYMBOL(foo); > > > > And module_bar: > > > > int bar(int y) > > { > > return 3 * baz(y); > > } > > EXPORT_SYMBOL(bar); > > > > And this in the main kernel image: > > > > int baz(int z) > > { > > return plonk(z); > > } > > EXPORT_SYMBOLbaz); > > > > Now we build the kernel and modules. Then we realize that nothing > > references symbol "foo". We can trim the "foo" export. But it would be > > necessary to recompile module_foo with LTO (especially if there is > > some other code in that module) to realize that nothing > > references foo() any longer and optimize away the reference to bar(). > > But, does LTO even do that to modules? Sure, the export metadata for foo > vanishes, so there's no function pointer reference to foo, but given > that modules are just -r links, the compiler/linker can't really assume > that the generated object won't later be linked with something that does > require foo? At least for the simpler case of --gc-sections, ld docs say: > > '--gc-sections' > ... > > This option can be set when doing a partial link (enabled with > option '-r'). In this case the root of symbols kept must be > explicitly specified either by one of the options '--entry', > '--undefined', or '--gc-keep-exported' or by a 'ENTRY' command in > the linker script. > > and I would assume that for LTO, --gc-keep-exported would be the only > sane semantics (keep any external symbol with default visibility). > > Can you point me at a tree/set of LTO patches and a toolchain where the > previous implementation would actually eventually eliminate bar() from > module_bar? All that I readily have is a link to the article I wrote with the results I obtained at the time: https://lwn.net/Articles/746780/. The toolchain and kernel tree are rather old at this point and some effort would be required to modernize everything. I don't remember if there was something special to do LTO on modules. Maybe Andi Kleen had something in his patchset for that: https://github.com/andikleen/linux-misc/blob/lto-415-2/Documentation/lto-build He mentions that LTO isn't very effective with modules enabled, but what I demonstrated in myarticle is that LTO becomes very effective with or without modules as long as CONFIG_TRIM_UNUSED_KSYMS is enabled. Having CONFIG_TRIM_UNUSED_KSYMS in one pass is likely to still be pretty effective even if possibly not not optimal. And maybe people don't really care for the missing 10% anyway (I'm just throwing a number in the air here). Nicolas
