On 25/02/2025 21:52, Andrii Nakryiko wrote: > On Tue, Feb 25, 2025 at 2:02 AM Alan Maguire <alan.maguire@xxxxxxxxxx> wrote: >> >> On 07/02/2025 23:50, Alexei Starovoitov wrote: >>> On Thu, Feb 6, 2025 at 5:21 PM Stephen Brennan >>> <stephen.s.brennan@xxxxxxxxxx> wrote: >>>> When the feature was implemented in pahole, my measurements indicated >>>> that vmlinux BTF size increased by about 25.8%, and module BTF size >>>> increased by 53.2%. Due to these increases, the feature is implemented >>>> behind a new config option, allowing users sensitive to increased memory >>>> usage to disable it. >>>> >>> >>> ... >>>> +config DEBUG_INFO_BTF_GLOBAL_VARS >>>> + bool "Generate BTF type information for all global variables" >>>> + default y >>>> + depends on DEBUG_INFO_BTF && PAHOLE_VERSION >= 128 >>>> + help >>>> + Include type information for all global variables in the BTF. This >>>> + increases the size of the BTF information, which increases memory >>>> + usage at runtime. With global variable types available, runtime >>>> + debugging and tracers may be able to provide more detail. >>> >>> This is not a solution. >>> Even if it's changed to 'default n' distros will enable it >>> like they enable everything and will suffer a regression. >>> >>> We need to add a new module like vmlinux_btf.ko that will contain >>> this additional BTF data. For global vars and everything else we might need. >>> >> >> In this area, I've been exploring adding support for >> CONFIG_DEBUG_INFO_BTF=m , so that the BTF info for vmlinux is delivered >> via a module. From the consumer side, everything looks identical >> (/sys/kernel/btf/vmlinux is there etc), it is just that the .BTF section >> is delivered via btf_vmlinux.ko instead. The original need for this was >> that embedded folks noted that because in the current situation BTF data >> is in vmlinux, they cannot enable BTF because such small-footprint >> systems do not support a large vmlinux binary. However they could >> potentially use kernel BTF if it was delivered via a module. The other >> nice thing about module delivery in the general case is we can make use >> of module compression. In experiments I see a 5.8Mb vmlinux BTF reduce >> to a 1.8Mb btf_vmlinux.ko.gz module on-disk. >> >> The challenge in delivering vmlinux BTF in a module is that on module >> load during boot other modules expect vmlinux BTF to be there when >> adding their own BTF to /sys/kernel/btf. And kfunc registration from >> kernel and modules expects this also. So support for deferred BTF module >> load/kfunc registration is required too. I've implemented the former and >> now am working on the latter. Hope to have some RFC patches ready soon, >> but it looks feasible at this point. > > Lazy btf_vmlinux.ko loading when BTF is actually needed (i.e., when > user reads /sys/kernel/btf/vmlinux for the first time; or when BPF > program is validated and needs kernel BTF) would be great. Curious too > see how all that fits together! > >> >> Assuming such an option was available to small-footprint systems, should >> we consider adding global variables to core vmlinux BTF along with >> per-cpu variables? Then vmlinux BTF extras could be used for some of the >> additional optional representations like function site-specific data >> (inlines etc)? Or are there other factors other than on-disk footprint >> that we need to consider? Thanks! > > I'd keep BTF for variables separate from "core" vmlinux BTF. We can > have /sys/kernel/btf/vmlinux.vars, which would depend on > /sys/kernel/btf/vmlinux as a base BTF. Separately, we could eventually > have /sys/kernel/btf/vmlinux.inlines which would also have > /sys/kernel/btf/vmlinux as base BTF. If no one needs vmlinux.vars on > the system, we won't need to waste memory on it. Seems more modular > and extensible. > Sounds good. So thinking about how this fits with CONFIG_DEBUG_INFO_BTF=m, perhaps the approach would be to use btf_vmlinux.ko for all such extensible /sys/kernel/btf/vmlinux.vars, vmlinux.inlines etc. Each of these is derived from .BTF.vars , .BTF.inlines sections in btf_vmlinux.ko. These are optionally included via CONFIG_DEBUG_INFO_BTF_EXTRAS list. If CONFIG_DEBUG_INFO_BTF=y the core vmlinux section stays in vmlinux itself and the extras are delivered via btf_vmlinux.ko, but if CONFIG_DEBUG_INFO_BTF=m, the vmlinux .BTF section is delivered in btf_vmlinux.ko too. If this makes sense, I'll try and put together the CONFIG_DEBUG_INFO_BTF=m support first, and that will give us a btf_vmlinux.ko to work with for delivery of extras. Thanks! Alan >> >> Alan >> >>> pw-bot: cr >>> >>
