On June 29, 2020 6:13:20 PM GMT-03:00, Hao Luo <haoluo@xxxxxxxxxx> wrote: >Hi Andrii, > >Thanks for taking a look! I can post a v5 based on your thorough and >detailed analysis this week. Some of my other work on sched consumed >most of my bandwidth last week and I very appreciate your help! I >apologize if it is felt I dropped this patch and will follow up within >this week. I'll also try and look at this tomorrow. - Arnaldo > >Hao > >On Mon, Jun 29, 2020 at 1:26 PM Andrii Nakryiko ><andrii.nakryiko@xxxxxxxxx> wrote: >> >> On Fri, Jun 19, 2020 at 1:30 PM Hao Luo <haoluo@xxxxxxxxxx> wrote: >> > >> > Hi, Andrii, >> > >> > I agree that we'd better put a hold on this patch until we find out >> > the reason for the 'unconventional' symbols. I'll _try_ to figure >it >> > out, but not able to fully commit my time on this patch. I thought >I'd >> > better publish this patch as the DATASEC and VARs are generated >> > correctly in format, so that anyone can use it to generate the >vmlinux >> > and continue the development on libbpf based on your ksym work >(i.e. >> > typed ksyms). >> > >> >> Hey Hao, >> >> It's a pity that you are willing to drop this while being half-step >> away from doing this properly. See my other reply, there is something >> fishy with variable__name() and variable__type_size() and how it >> calculates/caches values. ELF itself has all the data (based ELF >> symbol data) and it seems to be correct. And I was correct to be >> suspicious about just filtering out such variables, because in my >case >> you'd filter out a good chunk of variables for no good reason: 74 >> variables, which is a 26% of all per-CPU variables in my kernel. >> >> It's up to you, of course, but it would be nice to fix it up (just >> switching to elf_sym__size() and elf_sym__name() would be fine, you'd >> just need to make sure to add elf_sym_name() result into a string >> buffer; unless Arnaldo has some better alternatives) and publish >> complete v5, that would get merged into pahole. After that, building >> on my .ksym work in libbpf should get you to what you need from BPF >> pretty quickly (plus some kernel-side logic to recognize these >per-CPU >> variables). >> >> > Hao >> > >> > On Fri, Jun 19, 2020 at 12:58 PM Andrii Nakryiko >> > <andrii.nakryiko@xxxxxxxxx> wrote: >> > > >> > > On Thu, Jun 18, 2020 at 12:49 AM Hao Luo <haoluo@xxxxxxxxxx> >wrote: >> > > > >> > > > On SMP systems, the global percpu variables are placed in a >special >> > > > '.data..percpu' section, which is stored in a segment whose >initial >> > > > address is set to 0, the addresses of per-CPU variables are >relative >> > > > positive addresses [1]. >> > > > >> > > > This patch extracts these variables from vmlinux and places >them with >> > > > their type information in BTF. More specifically, when BTF is >encoded, >> > > > we find the index of the '.data..percpu' section and then >traverse >> > > > the symbol table to find those global objects which are in this >section. >> > > > For each of these objects, we push a BTF_KIND_VAR into the >types buffer, >> > > > and a BTF_VAR_SECINFO into another buffer, percpu_secinfo. When >all the >> > > > CUs have finished processing, we push a BTF_KIND_DATASEC into >the >> > > > btfe->types buffer, followed by the percpu_secinfo's content. >> > > > >> > > > In a v5.7-rc7 linux kernel, I was able to extract 291 such >variables. >> > > > The build time overhead is small and the space overhead is also >small. >> > > > >> > > > Testing: >> > > > >> > > > - vmlinux size has increased by ~12kb. >> > > > Before: >> > > > $ readelf -SW vmlinux | grep BTF >> > > > [25] .BTF PROGBITS ffffffff821a905c >13a905c 2d2bf8 00 >> > > > After: >> > > > $ pahole -J vmlinux >> > > > $ readelf -SW vmlinux | grep BTF >> > > > [25] .BTF PROGBITS ffffffff821a905c >13a905c 2d5bca 00 >> > > > >> > > > - Common global percpu VARs and DATASEC are found in BTF >section. >> > > > $ bpftool btf dump file vmlinux | grep runqueues >> > > > [14098] VAR 'runqueues' type_id=13725, linkage=global-alloc >> > > > >> > > > $ bpftool btf dump file vmlinux | grep 'cpu_stopper' >> > > > [17592] STRUCT 'cpu_stopper' size=72 vlen=5 >> > > > [17612] VAR 'cpu_stopper' type_id=17592, linkage=static >> > > > >> > > > $ bpftool btf dump file vmlinux | grep ' DATASEC ' >> > > > [63652] DATASEC '.data..percpu' size=0 vlen=294 >> > > >> > > probably forgot to update the example, I'd imagine size wouldn't >be 0 anymore? >> > > >> >> [...] -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
