On Tue, Feb 25, 2025 at 2:02 AM Alan Maguire <alan.maguire@xxxxxxxxxx> wrote: > > On 07/02/2025 23:50, Alexei Starovoitov wrote: > > On Thu, Feb 6, 2025 at 5:21 PM Stephen Brennan > > <stephen.s.brennan@xxxxxxxxxx> wrote: > >> When the feature was implemented in pahole, my measurements indicated > >> that vmlinux BTF size increased by about 25.8%, and module BTF size > >> increased by 53.2%. Due to these increases, the feature is implemented > >> behind a new config option, allowing users sensitive to increased memory > >> usage to disable it. > >> > > > > ... > >> +config DEBUG_INFO_BTF_GLOBAL_VARS > >> + bool "Generate BTF type information for all global variables" > >> + default y > >> + depends on DEBUG_INFO_BTF && PAHOLE_VERSION >= 128 > >> + help > >> + Include type information for all global variables in the BTF. This > >> + increases the size of the BTF information, which increases memory > >> + usage at runtime. With global variable types available, runtime > >> + debugging and tracers may be able to provide more detail. > > > > This is not a solution. > > Even if it's changed to 'default n' distros will enable it > > like they enable everything and will suffer a regression. > > > > We need to add a new module like vmlinux_btf.ko that will contain > > this additional BTF data. For global vars and everything else we might need. > > > > In this area, I've been exploring adding support for > CONFIG_DEBUG_INFO_BTF=m , so that the BTF info for vmlinux is delivered > via a module. From the consumer side, everything looks identical > (/sys/kernel/btf/vmlinux is there etc), it is just that the .BTF section > is delivered via btf_vmlinux.ko instead. The original need for this was > that embedded folks noted that because in the current situation BTF data > is in vmlinux, they cannot enable BTF because such small-footprint > systems do not support a large vmlinux binary. However they could > potentially use kernel BTF if it was delivered via a module. The other > nice thing about module delivery in the general case is we can make use > of module compression. In experiments I see a 5.8Mb vmlinux BTF reduce > to a 1.8Mb btf_vmlinux.ko.gz module on-disk. > > The challenge in delivering vmlinux BTF in a module is that on module > load during boot other modules expect vmlinux BTF to be there when > adding their own BTF to /sys/kernel/btf. And kfunc registration from > kernel and modules expects this also. So support for deferred BTF module > load/kfunc registration is required too. I've implemented the former and > now am working on the latter. Hope to have some RFC patches ready soon, > but it looks feasible at this point. Lazy btf_vmlinux.ko loading when BTF is actually needed (i.e., when user reads /sys/kernel/btf/vmlinux for the first time; or when BPF program is validated and needs kernel BTF) would be great. Curious too see how all that fits together! > > Assuming such an option was available to small-footprint systems, should > we consider adding global variables to core vmlinux BTF along with > per-cpu variables? Then vmlinux BTF extras could be used for some of the > additional optional representations like function site-specific data > (inlines etc)? Or are there other factors other than on-disk footprint > that we need to consider? Thanks! I'd keep BTF for variables separate from "core" vmlinux BTF. We can have /sys/kernel/btf/vmlinux.vars, which would depend on /sys/kernel/btf/vmlinux as a base BTF. Separately, we could eventually have /sys/kernel/btf/vmlinux.inlines which would also have /sys/kernel/btf/vmlinux as base BTF. If no one needs vmlinux.vars on the system, we won't need to waste memory on it. Seems more modular and extensible. > > Alan > > > pw-bot: cr > > >
