On Mon, Dec 30, 2024 at 01:07:36PM -0300, Arnaldo Carvalho de Melo wrote: > On Sat, Dec 21, 2024 at 01:23:38AM +0000, Ihor Solodrai wrote: > > Multithreading is now contained in dwarf_loader.c, and is implemented > > using a jobs queue and a pool of worker threads. As a consequence, > > multithreading-related code is removed from pahole.c. > > > > A single-thread special case is removed: queueing setup works fine > > with a single worker, which will switch between jobs as appropriate. > > > > Code supporting previous version of the multithreading, such as > > cu_state, thread_data and related functions, is also removed. > > > > reproducible_build flag is now moot: the BTF encoding is always > > reproducible with these changes. > > > > The goal outlined in the RFC [1] - making parallel reproducible BTF > > generation as fast as non-reproducible - is achieved by implementing > > the requirement of ordered CU encoding (stealing) directly in the job > > queue in dwarf_loader.c > > > > The synchronization in the queue is implemented by a mutex (which > > ensures consistency of queue state) and a job_added condition > > variable. Motivation behind using condition variables is a classic > > one: we want to avoid the threads checking the state of the queue in a > > busy loop, competing for a single mutex. > > > > In comparison to the previous version of this patch [2], job_taken > > condition variable is removed. The number of decoded CUs in memory is > > now limited by initial JOB_DECODE jobs. The enqueue/dequeue interface > > is changed aiming to reduce locking. See relevant discussion [3]. > > So I'm running tests/tests after each of these patches and at this point > I got: > > root@number:/home/acme/git/pahole# tests/tests > 1: Validation of BTF encoding of functions; this may take some time: grep: /tmp/btf_functions.sh.OgxoO4/dwarf.funcs: binary file matches > ERROR: mismatch : BTF 'bool evtchn_fifo_is_pending(evtchn_port_t);' not found; DWARF '' > Test data is in /tmp/btf_functions.sh.OgxoO4 > 2: Default BTF on a system without BTF: Ok > 3: Flexible arrays accounting: Ok > 4: Check that pfunct can print btf_decl_tags read from BTF: Ok > 5: Pretty printing of files using DWARF type information: Ok > 6: Parallel reproducible DWARF Loading/Serial BTF encoding: Ok > /home/acme/git/pahole > root@number:/home/acme/git/pahole# > > root@number:/home/acme/git/pahole# grep evtchn_fifo_is_pending /tmp/btf_functions.sh.OgxoO4/dwarf.funcs > bool evtchn_fifo_is_pending(evtchn_port_t); > root@number:/home/acme/git/pahole# grep evtchn_fifo_is_pending /tmp/btf_functions.sh.OgxoO4/btf.funcs > bool evtchn_fifo_is_pending(evtchn_port_t); > root@number:/home/acme/git/pahole# > > Which seems like a test artifact: > > root@number:/home/acme/git/pahole# pfunct -f evtchn_fifo_is_pending /tmp/btf_functions.sh.OgxoO4/vmlinux.btf > bool evtchn_fifo_is_pending(evtchn_port_t port); > root@number:/home/acme/git/pahole# pfunct -F btf -f evtchn_fifo_is_pending /tmp/btf_functions.sh.OgxoO4/vmlinux.btf > bool evtchn_fifo_is_pending(evtchn_port_t port); > root@number:/home/acme/git/pahole# > > root@number:/home/acme/git/pahole# pfunct -F btf -f evtchn_fifo_is_pending /lib/modules/6.13.0-rc2/build/vmlinux > bool evtchn_fifo_is_pending(evtchn_port_t port); > root@number:/home/acme/git/pahole# pfunct -F dwarf evtchn_fifo_is_pending /lib/modules/6.13.0-rc2/build/vmlinux > bool evtchn_fifo_is_pending(evtchn_port_t port); > bool evtchn_fifo_is_pending(evtchn_port_t port); > root@number:/home/acme/git/pahole# > > Maybe because DWARF has two such functions? Alan? > > But: > > root@number:/home/acme/git/pahole# readelf -wi /lib/modules/6.13.0-rc2/build/vmlinux | grep -B2 -A12 evtchn_fifo_is_pending > <2><946502d>: Abbrev Number: 0 > <1><946502e>: Abbrev Number: 128 (DW_TAG_subprogram) > <9465030> DW_AT_name : (indirect string, offset: 0x39779e): evtchn_fifo_is_pending > <9465034> DW_AT_decl_file : 1 > <9465034> DW_AT_decl_line : 206 > <9465035> DW_AT_decl_column : 13 > <9465036> DW_AT_prototyped : 1 > <9465036> DW_AT_type : <0x9456abf> > <946503a> DW_AT_low_pc : 0xffffffff81b6d330 > <9465042> DW_AT_high_pc : 0x65 > <946504a> DW_AT_frame_base : 1 byte block: 9c (DW_OP_call_frame_cfa) > <946504c> DW_AT_call_all_calls: 1 > <946504c> DW_AT_sibling : <0x946517a> > <2><9465050>: Abbrev Number: 81 (DW_TAG_formal_parameter) > <9465051> DW_AT_name : (indirect string, offset: 0x3abde1): port > root@number:/home/acme/git/pahole# > > I.e. there is just one, but when pfunct prints it must be seeing some > other usage that refers back to 946502e... > > root@number:/home/acme/git/pahole# readelf -wi /lib/modules/6.13.0-rc2/build/vmlinux | grep -w -B2 -A12 0x946502e > <4><9464825>: Abbrev Number: 60 (DW_TAG_call_site) > <9464826> DW_AT_call_return_pc: 0xffffffff81b6d4f5 > <946482e> DW_AT_call_origin : <0x946502e> > <9464832> DW_AT_sibling : <0x946483d> > <5><9464836>: Abbrev Number: 13 (DW_TAG_call_site_parameter) > <9464837> DW_AT_location : 1 byte block: 55 (DW_OP_reg5 (rdi)) > <9464839> DW_AT_call_value : 2 byte block: 7f 0 (DW_OP_breg15 (r15): 0) > <5><946483c>: Abbrev Number: 0 > <4><946483d>: Abbrev Number: 60 (DW_TAG_call_site) > <946483e> DW_AT_call_return_pc: 0xffffffff81b6d590 > <9464846> DW_AT_call_origin : <0x9463caa> > <946484a> DW_AT_sibling : <0x946485d> > <5><946484e>: Abbrev Number: 13 (DW_TAG_call_site_parameter) > <946484f> DW_AT_location : 1 byte block: 55 (DW_OP_reg5 (rdi)) > <9464851> DW_AT_call_value : 2 byte block: 7f 0 (DW_OP_breg15 (r15): 0) > -- > <9464f6b> DW_AT_frame_base : 1 byte block: 9c (DW_OP_call_frame_cfa) > <9464f6d> DW_AT_call_all_calls: 1 > <9464f6d> DW_AT_sibling : <0x946502e> > <2><9464f71>: Abbrev Number: 81 (DW_TAG_formal_parameter) > <9464f72> DW_AT_name : (indirect string, offset: 0x3abde1): port > <9464f76> DW_AT_decl_file : 1 > <9464f76> DW_AT_decl_line : 212 > <9464f77> DW_AT_decl_column : 44 > <9464f78> DW_AT_type : <0x9463263> > <9464f7c> DW_AT_location : 0x181ea81 (location list) > <9464f80> DW_AT_GNU_locviews: 0x181ea79 > <2><9464f84>: Abbrev Number: 64 (DW_TAG_variable) > <9464f85> DW_AT_name : (indirect string, offset: 0x9f2cd): word > <9464f89> DW_AT_decl_file : 1 > <9464f89> DW_AT_decl_line : 214 > root@number:/home/acme/git/pahole# > > Not really :-\ > > root@number:/home/acme/git/pahole# pfunct --decl_info -F dwarf evtchn_fifo_is_pending /lib/modules/6.13.0-rc2/build/vmlinux > /* Used at: /home/acme/git/linux/drivers/xen/events/events_fifo.c */ > /* <946502e> /home/acme/git/linux/drivers/xen/events/events_fifo.c:206 */ > bool evtchn_fifo_is_pending(evtchn_port_t port); > /* Used at: /home/acme/git/linux/drivers/xen/events/events_fifo.c */ > /* <946502e> /home/acme/git/linux/drivers/xen/events/events_fifo.c:206 */ > bool evtchn_fifo_is_pending(evtchn_port_t port); > root@number:/home/acme/git/pahole# > > So far I couldn't find an explanation for this oddity... Lets see if > after applying all patches we get past this. Its not related to this patch series, before we get two outputs for these (and other functions in /home/acme/git/linux/drivers/xen/events/events_fifo.c). Still investigating. - Arnaldo
