Hi,First of all, many thanks for your time. Did you say that the first file has problems?
The first file (http://gofile.io/?c=qk6oXv) has repeated gmallocn()s while the second (https://gofile.io/?c=oGxgSM) also has problems with unmatched (not necessarily repeated) function calls. I am not sure if the kernel for the second one is 5.4.7 or the generic Ubuntu kernel. But the first one is certainly 5.4.7. Just to be clear, there were many instances of these unmatched <caller, callees>. I have a simple python script that checks for this situation. It disassembles functions using GDB and checks the (directly called) target of each caller. I will put some comments in the script and upload it. Could you check to see if the python script detects any mismatches in your backtraces? It takes the perf script output file as input. I will upload the script in an hour.
Regards. On 2020-03-26 14:29, Jiri Olsa wrote:
On Thu, Mar 26, 2020 at 03:39:31AM +0430, ahmadkhorrami wrote:An here is the second one: https://gofile.io/?c=oGxgSMthanks, so far I don't see that, but I think it's because the 'inline' code does not resolve the libc dso correctly, CC-ing few other folks.. so I'm able to get:EvJobScheduler 17382 13006.872877: 10000 mem_load_uops_retired.l3_miss:uppp: 7fffd2e06588 5080022 N/A|SNP N/A|TLB N/A|LCK N/A7ffff4b04c74 [unknown] (/lib/x86_64-linux-gnu/libc-2.27.so) 7ffff4b072ec malloc+0x27c (/lib/x86_64-linux-gnu/libc-2.27.so)7fffd9872ddd gmalloc+0xd (/usr/lib/x86_64-linux-gnu/libpoppler.so.73.0.0) 7fffd9873391 copyString+0x11 (/usr/lib/x86_64-linux-gnu/libpoppler.so.73.0.0)while you see:EvJobScheduler 17382 13006.872877: 10000 mem_load_uops_retired.l3_miss:uppp: 7fffd2e06588 5080022 N/A|SNP N/A|TLB N/A|LCK N/A7ffff4b04c74 _int_malloc+0x9a4 (/lib/x86_64-linux-gnu/libc-2.27.so) 7ffff4b072ec __GI___libc_malloc+0x27c (inlined)7fffd9872ddd gmalloc+0xd (/usr/lib/x86_64-linux-gnu/libpoppler.so.73.0.0) 7fffd9872ddd gmalloc+0xd (/usr/lib/x86_64-linux-gnu/libpoppler.so.73.0.0)so for some reason I don't resolve 7ffff4b04c74, which might be the reason I don't see the following address twice as you do:7fffd9872ddd gmalloc+0xd (/usr/lib/x86_64-linux-gnu/libpoppler.so.73.0.0) 7fffd9872ddd gmalloc+0xd (/usr/lib/x86_64-linux-gnu/libpoppler.so.73.0.0)the previous field (7ffff4b072ec) is resolved by the 'inline' code, so I wonder it's related I needed to make some changes to utils/srcline.c to be able to properly open dso via buildid cache, I pushed it to: git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git perf/callchain here are the steps to get above output: download perf.data and perf archive data from: https://gofile.io/?c=o95O7N http://gofile.io/?c=qk6oXv $ unxz ./perf.data.xz $ tar xvf perf.data.tar.bz2 -C ~/.debug compile perf from aobve tree/branch and run: $ perf script -i perf.data I think we might be missing some libraries in 'perf archive' by not following debug_link section, will check on this jirka Regards On 2020-03-26 02:51, ahmadkhorrami wrote: Here is the link for the gmallocn()s: http://gofile.io/?c=qk6oXv I will send the second one as soon as the upload is finished: Regards. On 2020-03-26 02:16, Jiri Olsa wrote: On Thu, Mar 26, 2020 at 02:07:39AM +0430, ahmadkhorrami wrote: Here is the link for the gmallocn() case: https://gofile.io/?c=o95O7N The output for the second case is big. I have a small one produced several days ago the link of which is as follows: https://gofile.io/?c=OIPCjx looking good, but I still need you t run 'perf archive' on top of your data and send me the perf.data.tar.bz2 it generates, like: [jolsa@krava perf]$ sudo ./perf record ^C[ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 1.675 MB perf.data (6248 samples) ] [jolsa@krava perf]$ sudo ./perf archive Now please run: $ tar xvf perf.data.tar.bz2 -C ~/.debug wherever you need to run 'perf report' on. I need that perf.data.tar.bz2 generated from your data thanks, jirka Regards. On 2020-03-26 01:39, Steven Rostedt wrote: On Wed, 25 Mar 2020 22:02:52 +0100 Jiri Olsa <jolsa@xxxxxxxxxx> wrote: yea, no luck.. so if you could generate some reasonable small perf.data that shows the issue and send it over together with 'perf archive' data privately to me and to whoever else ask for it, so we don't polute the list.. Right. And it may be better if you compress it too. xz perf.data and attach the perf.data.xz (and only privately send it to Mr. Olsa). -- Steve or if you could put it somewhere on the web/ftp.. that'd be best
|