Em Wed, Aug 05, 2020 at 03:06:04PM -0300, Arnaldo Carvalho de Melo escreveu: > Em Wed, Aug 05, 2020 at 10:07:58AM -0700, Hao Luo escreveu: > > Friendly ping. Do you have a timeline for merging this feature? Is > > there anything that I can help? > I had already merged it, but wasn't in the master branch, it is now, I'm > finishing some other feature to then cut v1.18, I just committed https://git.kernel.org/pub/scm/devel/pahole/pahole.git/commit/?h=pretty&id=22f93766cf02f4e0bda628ae55af07063b74176a I think this is the time to cut a new version, I'll do some more testing and friday release v1.18. If you could please retest with what is in the 'pretty' branch, which includes your patch and these raw pretty printing new feature, that would be awesome. I don't see how these pretty printing changes would disrupt what you did, as btf encoding is done before the codepaths for pretty printing, be it the types or raw data according to the types. Ah, at some point in the future, maybe for v1.19, we can use the new information you're encoding in BTF to pretty print the content of those variables :-) Best regards, - Arnaldo > Thanks, > > - Arnaldo > > > Thanks, > > Hao > > > > > > On Thu, Jul 9, 2020 at 11:17 AM Hao Luo <haoluo@xxxxxxxxxx> wrote: > > > > > > Sounds good. Thanks! > > > > > > On Thu, Jul 9, 2020 at 8:41 AM Arnaldo Carvalho de Melo <acme@xxxxxxxxxx> wrote: > > > > > > > > Em Wed, Jul 08, 2020 at 01:56:02PM -0700, Andrii Nakryiko escreveu: > > > > > On Wed, Jul 8, 2020 at 1:44 PM Hao Luo <haoluo@xxxxxxxxxx> wrote: > > > > > > > > > > > > On SMP systems, the global percpu variables are placed in a special > > > > > > '.data..percpu' section, which is stored in a segment whose initial > > > > > > address is set to 0, the addresses of per-CPU variables are relative > > > > > > positive addresses [1]. > > > > > > > > > > > > This patch extracts these variables from vmlinux and places them with > > > > > > their type information in BTF. More specifically, when BTF is encoded, > > > > > > we find the index of the '.data..percpu' section and then traverse > > > > > > the symbol table to find those global objects which are in this section. > > > > > > For each of these objects, we push a BTF_KIND_VAR into the types buffer, > > > > > > and a BTF_VAR_SECINFO into another buffer, percpu_secinfo. When all the > > > > > > CUs have finished processing, we push a BTF_KIND_DATASEC into the > > > > > > btfe->types buffer, followed by the percpu_secinfo's content. > > > > > > > > > > > > In a v5.8-rc3 linux kernel, I was able to extract 288 such variables. > > > > > > The build time overhead is small and the space overhead is also small. > > > > > > See testings below. > > > > > > > > > > > > A found variable can be invalid in two ways: > > > > > > > > > > > > - Its name found in elf_sym__name is invalid. > > > > > > - Its size identified by elf_sym__size is 0. > > > > > > > > > > > > In either case, the btf containing such symbols will be rejected by the > > > > > > btf verifier. Normally we should not see such symbols. But if one is > > > > > > seen during btf encoding, the encoder will exit with error. An new flag > > > > > > '-j' (or '--force') is implemented to help testing, which skips the > > > > > > invalid symbols and force emit a btf. > > > > > > > > > > > > Testing: > > > > > > > > > > > > - Vmlinux size has increased by ~12kb. > > > > > > Before: > > > > > > $ readelf -SW vmlinux | grep BTF > > > > > > [25] .BTF PROGBITS ffffffff821a905c 13a905c 2d2bf8 00 > > > > > > After: > > > > > > $ pahole -J vmlinux > > > > > > $ readelf -SW vmlinux | grep BTF > > > > > > [25] .BTF PROGBITS ffffffff821a905c 13a905c 2d5bca 00 > > > > > > > > > > > > - Common global percpu VARs and DATASEC are found in BTF section. > > > > > > $ bpftool btf dump file vmlinux | grep runqueues > > > > > > [14152] VAR 'runqueues' type_id=13778, linkage=global-alloc > > > > > > > > > > > > $ bpftool btf dump file vmlinux | grep 'cpu_stopper' > > > > > > [17582] STRUCT 'cpu_stopper' size=72 vlen=5 > > > > > > [17601] VAR 'cpu_stopper' type_id=17582, linkage=static > > > > > > > > > > > > $ bpftool btf dump file vmlinux | grep ' DATASEC ' > > > > > > [63652] DATASEC '.data..percpu' size=179288 vlen=288 > > > > > > > > > > > > - Tested bpf selftests. > > > > > > > > > > > > - Pahole exits with error if an invalid symbol is seen during encoding, > > > > > > make -f Makefile -j 36 -s > > > > > > PAHOLE: Error: Found symbol of zero size when encoding btf (sym: 'yyy', cu: 'xxx.c'). > > > > > > PAHOLE: Error: Use '-j' or '--force_emit' to ignore such symbols and force emit the btf. > > > > > > scripts/link-vmlinux.sh: line 137: 2475712 Segmentation fault LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1} > > > > > > > > > > > > - With the flag '-j' or '--force', the invalid symbols are ignored. > > > > > > > > > > > > - Further in verbose mode and with '-j' or '--force' set, a warning is generated: > > > > > > PAHOLE: Warning: Found symbol of zero size when encoding btf, ignored (sym: 'yyy', cu: 'xxx.c'). > > > > > > PAHOLE: Warning: Found symbol of invalid name when encoding btf, ignored (sym: 'zzz', cu: 'sss.c'). > > > > > > > > > > > > References: > > > > > > [1] https://lwn.net/Articles/531148/ > > > > > > > > > > > > Acked-by: Andrii Nakryiko <andriin@xxxxxx> > > > > > > Tested-by: Andrii Nakryiko <andriin@xxxxxx> > > > > > > Signed-off-by: Hao Luo <haoluo@xxxxxxxxxx> > > > > > > --- > > > > > > > > > > Looks great, thanks! > > > > > > > > The only thing I didn't like was to grab a one letter option for > > > > something that should be seldomly used, so I'll make it be just --force, > > > > leaving -j for future usage, > > > > > > > > I'll test this, make this change and apply, probably this, together with > > > > some pretty printing features I'm working on are enough for a new > > > > release. > > > > > > > > Thanks, > > > > > > > > - Arnaldo > > -- > > - Arnaldo -- - Arnaldo
