On Sat, Dec 21, 2024 at 01:23:38AM +0000, Ihor Solodrai wrote:
> Multithreading is now contained in dwarf_loader.c, and is implemented
> using a jobs queue and a pool of worker threads. As a consequence,
> multithreading-related code is removed from pahole.c.
>
> A single-thread special case is removed: queueing setup works fine
> with a single worker, which will switch between jobs as appropriate.
>
> Code supporting previous version of the multithreading, such as
> cu_state, thread_data and related functions, is also removed.
>
> reproducible_build flag is now moot: the BTF encoding is always
> reproducible with these changes.
>
> The goal outlined in the RFC [1] - making parallel reproducible BTF
> generation as fast as non-reproducible - is achieved by implementing
> the requirement of ordered CU encoding (stealing) directly in the job
> queue in dwarf_loader.c
>
> The synchronization in the queue is implemented by a mutex (which
> ensures consistency of queue state) and a job_added condition
> variable. Motivation behind using condition variables is a classic
> one: we want to avoid the threads checking the state of the queue in a
> busy loop, competing for a single mutex.
>
> In comparison to the previous version of this patch [2], job_taken
> condition variable is removed. The number of decoded CUs in memory is
> now limited by initial JOB_DECODE jobs. The enqueue/dequeue interface
> is changed aiming to reduce locking. See relevant discussion [3].
So I'm running tests/tests after each of these patches and at this point
I got:
root@number:/home/acme/git/pahole# tests/tests
1: Validation of BTF encoding of functions; this may take some time: grep: /tmp/btf_functions.sh.OgxoO4/dwarf.funcs: binary file matches
ERROR: mismatch : BTF 'bool evtchn_fifo_is_pending(evtchn_port_t);' not found; DWARF ''
Test data is in /tmp/btf_functions.sh.OgxoO4
2: Default BTF on a system without BTF: Ok
3: Flexible arrays accounting: Ok
4: Check that pfunct can print btf_decl_tags read from BTF: Ok
5: Pretty printing of files using DWARF type information: Ok
6: Parallel reproducible DWARF Loading/Serial BTF encoding: Ok
/home/acme/git/pahole
root@number:/home/acme/git/pahole#
root@number:/home/acme/git/pahole# grep evtchn_fifo_is_pending /tmp/btf_functions.sh.OgxoO4/dwarf.funcs
bool evtchn_fifo_is_pending(evtchn_port_t);
root@number:/home/acme/git/pahole# grep evtchn_fifo_is_pending /tmp/btf_functions.sh.OgxoO4/btf.funcs
bool evtchn_fifo_is_pending(evtchn_port_t);
root@number:/home/acme/git/pahole#
Which seems like a test artifact:
root@number:/home/acme/git/pahole# pfunct -f evtchn_fifo_is_pending /tmp/btf_functions.sh.OgxoO4/vmlinux.btf
bool evtchn_fifo_is_pending(evtchn_port_t port);
root@number:/home/acme/git/pahole# pfunct -F btf -f evtchn_fifo_is_pending /tmp/btf_functions.sh.OgxoO4/vmlinux.btf
bool evtchn_fifo_is_pending(evtchn_port_t port);
root@number:/home/acme/git/pahole#
root@number:/home/acme/git/pahole# pfunct -F btf -f evtchn_fifo_is_pending /lib/modules/6.13.0-rc2/build/vmlinux
bool evtchn_fifo_is_pending(evtchn_port_t port);
root@number:/home/acme/git/pahole# pfunct -F dwarf evtchn_fifo_is_pending /lib/modules/6.13.0-rc2/build/vmlinux
bool evtchn_fifo_is_pending(evtchn_port_t port);
bool evtchn_fifo_is_pending(evtchn_port_t port);
root@number:/home/acme/git/pahole#
Maybe because DWARF has two such functions? Alan?
But:
root@number:/home/acme/git/pahole# readelf -wi /lib/modules/6.13.0-rc2/build/vmlinux | grep -B2 -A12 evtchn_fifo_is_pending
<2><946502d>: Abbrev Number: 0
<1><946502e>: Abbrev Number: 128 (DW_TAG_subprogram)
<9465030> DW_AT_name : (indirect string, offset: 0x39779e): evtchn_fifo_is_pending
<9465034> DW_AT_decl_file : 1
<9465034> DW_AT_decl_line : 206
<9465035> DW_AT_decl_column : 13
<9465036> DW_AT_prototyped : 1
<9465036> DW_AT_type : <0x9456abf>
<946503a> DW_AT_low_pc : 0xffffffff81b6d330
<9465042> DW_AT_high_pc : 0x65
<946504a> DW_AT_frame_base : 1 byte block: 9c (DW_OP_call_frame_cfa)
<946504c> DW_AT_call_all_calls: 1
<946504c> DW_AT_sibling : <0x946517a>
<2><9465050>: Abbrev Number: 81 (DW_TAG_formal_parameter)
<9465051> DW_AT_name : (indirect string, offset: 0x3abde1): port
root@number:/home/acme/git/pahole#
I.e. there is just one, but when pfunct prints it must be seeing some
other usage that refers back to 946502e...
root@number:/home/acme/git/pahole# readelf -wi /lib/modules/6.13.0-rc2/build/vmlinux | grep -w -B2 -A12 0x946502e
<4><9464825>: Abbrev Number: 60 (DW_TAG_call_site)
<9464826> DW_AT_call_return_pc: 0xffffffff81b6d4f5
<946482e> DW_AT_call_origin : <0x946502e>
<9464832> DW_AT_sibling : <0x946483d>
<5><9464836>: Abbrev Number: 13 (DW_TAG_call_site_parameter)
<9464837> DW_AT_location : 1 byte block: 55 (DW_OP_reg5 (rdi))
<9464839> DW_AT_call_value : 2 byte block: 7f 0 (DW_OP_breg15 (r15): 0)
<5><946483c>: Abbrev Number: 0
<4><946483d>: Abbrev Number: 60 (DW_TAG_call_site)
<946483e> DW_AT_call_return_pc: 0xffffffff81b6d590
<9464846> DW_AT_call_origin : <0x9463caa>
<946484a> DW_AT_sibling : <0x946485d>
<5><946484e>: Abbrev Number: 13 (DW_TAG_call_site_parameter)
<946484f> DW_AT_location : 1 byte block: 55 (DW_OP_reg5 (rdi))
<9464851> DW_AT_call_value : 2 byte block: 7f 0 (DW_OP_breg15 (r15): 0)
--
<9464f6b> DW_AT_frame_base : 1 byte block: 9c (DW_OP_call_frame_cfa)
<9464f6d> DW_AT_call_all_calls: 1
<9464f6d> DW_AT_sibling : <0x946502e>
<2><9464f71>: Abbrev Number: 81 (DW_TAG_formal_parameter)
<9464f72> DW_AT_name : (indirect string, offset: 0x3abde1): port
<9464f76> DW_AT_decl_file : 1
<9464f76> DW_AT_decl_line : 212
<9464f77> DW_AT_decl_column : 44
<9464f78> DW_AT_type : <0x9463263>
<9464f7c> DW_AT_location : 0x181ea81 (location list)
<9464f80> DW_AT_GNU_locviews: 0x181ea79
<2><9464f84>: Abbrev Number: 64 (DW_TAG_variable)
<9464f85> DW_AT_name : (indirect string, offset: 0x9f2cd): word
<9464f89> DW_AT_decl_file : 1
<9464f89> DW_AT_decl_line : 214
root@number:/home/acme/git/pahole#
Not really :-\
root@number:/home/acme/git/pahole# pfunct --decl_info -F dwarf evtchn_fifo_is_pending /lib/modules/6.13.0-rc2/build/vmlinux
/* Used at: /home/acme/git/linux/drivers/xen/events/events_fifo.c */
/* <946502e> /home/acme/git/linux/drivers/xen/events/events_fifo.c:206 */
bool evtchn_fifo_is_pending(evtchn_port_t port);
/* Used at: /home/acme/git/linux/drivers/xen/events/events_fifo.c */
/* <946502e> /home/acme/git/linux/drivers/xen/events/events_fifo.c:206 */
bool evtchn_fifo_is_pending(evtchn_port_t port);
root@number:/home/acme/git/pahole#
So far I couldn't find an explanation for this oddity... Lets see if
after applying all patches we get past this.
- Arnaldo
