Hi, After enabling DEBUG_INFO in kernel I still get this error for BPF test. Please help. # perf test BPF -v ..... Looking at the vmlinux_path (8 entries long) Using /usr/lib/debug/boot/vmlinux-4.9.00-xxxx-amd-x86-64-00071-gd94c220-dirty for symbols Open Debuginfo file: /usr/lib/debug/boot/vmlinux-4.9.00-xxxx-amd-x86-64-00071-gd94c220-dirty Try to find probe point from debuginfo. Matched function: null_lseek [105be32] Probe point found: null_lseek+0 Searching 'file' variable in context. Converting variable file into trace event. converting f_mode in file file(type:file) has no member f_mode. An error occurred in debuginfo analysis (-22). bpf_probe: failed to convert perf probe eventsFailed to add events selected by BPF test child finished with -1 ---- end ---- Test BPF filter subtest 1: FAILED! On Thu, Feb 1, 2018 at 10:50 AM, Pintu Kumar <pintu.ping@xxxxxxxxx> wrote: > Dear Masami, > > Now I am stuck again with 'perf test' failure on 4.9 > > # perf --version > perf version 4.9.20-<product name> > > # perf test > 16: Try 'import perf' in python, checking link problems : FAILED! > 37.2: Test BPF prologue generation : FAILED! > > If you have any clue about these failure please hep me. > > Here are the verbose output: > ----------------------------------------------------------------------------- > 1) # perf test python -v > 16: Try 'import perf' in python, checking link problems : > --- start --- > test child forked, pid 24562 > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > ImportError: No module named perf > test child finished with -1 > ---- end ---- > Try 'import perf' in python, checking link problems: FAILED! > ------------------------------------------------------------------------------ > > 2) # perf test BPF -v > ------------------------------------------------------------------------------- > ..... > bpf: config 'func=null_lseek file->f_mode offset orig' is ok > Looking at the vmlinux_path (8 entries long) > symsrc__init: cannot get elf header. > Failed to find the path for kernel: Invalid ELF file > bpf_probe: failed to convert perf probe eventsFailed to add events > selected by BPF > test child finished with -1 > ---- end ---- > Test BPF filter subtest 1: FAILED! > > ------------------------------------------------------------------------------- > > > Thanks, > Pintu > > > On Wed, Jan 31, 2018 at 9:01 AM, Masami Hiramatsu <mhiramat@xxxxxxxxxx> wrote: >> On Tue, 30 Jan 2018 19:20:36 +0530 >> Pintu Kumar <pintu.ping@xxxxxxxxx> wrote: >> >>> On Tue, Jan 30, 2018 at 11:13 AM, Masami Hiramatsu <mhiramat@xxxxxxxxxx> wrote: >>> > >>> > On Mon, 29 Jan 2018 22:00:52 +0530 >>> > Pintu Kumar <pintu.ping@xxxxxxxxx> wrote: >>> > >>> > > Dear Masami, >>> > > >>> > > Thank you so much for your reply. >>> > > Please find some of my answers inline. >>> > > >>> > > >>> > > On Mon, Jan 29, 2018 at 7:47 PM, Masami Hiramatsu <mhiramat@xxxxxxxxxx> wrote: >>> > > > On Mon, 29 Jan 2018 13:40:34 +0530 >>> > > > Pintu Kumar <pintu.ping@xxxxxxxxx> wrote: >>> > > > >>> > > >> Hi All, >>> > > >> >>> > > >> 'perf probe' is failing sometimes on 4.9.20 with AMD-64. >>> > > >> # perf probe --add schedule >>> > > >> schedule is out of .text, skip it. >>> > > >> Error: Failed to add events. >>> > > >> >>> > > >> If any one have come across this problem please let me know the cause. >>> > > > >>> > > > Hi Pintu, >>> > > > >>> > > > Could you run it with --vv? >>> > > > >>> > > Ok, I will send verbose output by tomorrow. >>> > > >>> > > >> >>> > > >> Note: I don't have CONFIG_DEBUG_INFO enabled in kernel. Is this the problem? >>> > > > >>> > > > Without it, you can not probe source-level probe nor trace local variable. >>> > > > >>> > > >>> > > Currently I am facing problem in enabling DEBUG_INFO in our kernel 4.9.20 >>> > > However, I will try to manually include "-g" option during compilation. >>> > > >>> > > >> However, I manually copied the vmlinux file to /boot/ directory, but >>> > > >> still it does not work. >>> > > > >>> > > > That doesn't work. >>> > > > CONFIG_DEBUG_INFO option enables gcc to compile kernel with extra debuginfo. >>> > > > Without that option, debuginfo is not generated with vmlinux. >>> > > > >>> > > >> >>> > > >> I checked upstream patches until 4.15 but could not find any clue. >>> > > >> Please let me know if there is any fixes available for this. >>> > > > >>> > > > Could you also ensure that you run perf by root user? >>> > > > >>> > > >>> > > Yes I am running with root user. >>> > > >>> > > My concern is sometimes it works but sometimes it fails. >>> > >>> > What I thought was that your kernel enables kptr_strict(but maybe not.) >>> > Can you also try to find "schedule" and "_etext" functions in >>> > /proc/kallsyms? >>> > >>> >>> Yes these functions are present in kallsyms. >>> >>> # cat /proc/kallsyms | grep -w schedule ; cat /proc/kallsyms | grep -w _etext >>> ffffffff829328e0 T schedule >>> ffffffff8293e751 T _etext >>> >>> >>> > > I still needs to figure out, in which condition it works and which >>> > > condition it fails. >>> > >>> > Yeah, it is important. >>> > >>> > > Usually I noticed that in fresh reboot case it works. >>> > >>> > So, after a while, it doesn't work again? If so, it sounds like a daemon >>> > process changes settings in background. >>> > >>> >>> Fortunately or unfortunately, now it is passing every time. >>> Still not sure about the cause. >>> If it fails again, I will collect info with -v and report again. >>> Any specific info that I need to check when it fails ? >> >> OK, if you hit it again, please run perf-probe with -vvv option >> (-v is enough usually, if it is unknown bug, -vvv will be better) >> >> Thank you, >> >> >>> >>> Thank You, >>> Pintu >>> >>> >>> > Thank you, >>> > >>> > > >>> > > >>> > > > Thank you, >>> > > > >>> > > > >>> > > >> >>> > > >> >>> > > >> Thank You! >>> > > >> Regards, >>> > > >> Pintu >>> > > >> >>> > > >> >>> > > >> On Thu, Jan 25, 2018 at 7:09 PM, Pintu Kumar <pintu.ping@xxxxxxxxx> wrote: >>> > > >> > Hi, >>> > > >> > >>> > > >> > ** Changed the subject now, since these issues are related to general >>> > > >> > perf commands. >>> > > >> > >>> > > >> > Following are the issues: >>> > > >> > >>> > > >> > 1) perf probe --add schedule - FAILED >>> > > >> > output: >>> > > >> > schedule is out of .text, skip it. >>> > > >> > Error: Failed to add events. >>> > > >> > >>> > > >> > what is the issue here? >>> > > >> > Sometimes it pass and sometimes it fails... >>> > > >> > Similar is the case of 'perf inject' as well. >>> > > >> > >>> > > >> > 2) perf test - 1 FAILURE >>> > > >> > 37.1: Test basic BPF filtering : FAILED! >>> > > >> > 37.2: Test BPF prologue generation : Skip >>> > > >> > 37.3: Test BPF relocation checker : Skip >>> > > >> > >>> > > >> > bpf: config program 'func=SyS_epoll_wait' >>> > > >> > symbol:SyS_epoll_wait file:(null) line:0 offset:0 return:0 lazy:(null) >>> > > >> > bpf: config 'func=SyS_epoll_wait' is ok >>> > > >> > Looking at the vmlinux_path (8 entries long) >>> > > >> > Using /boot/vmlinux for symbols >>> > > >> > Could not open debuginfo. Try to use symbols. >>> > > >> > SyS_epoll_wait is out of .text, skip it. >>> > > >> > bpf_probe: failed to convert perf probe eventsFailed to add events >>> > > >> > selected by BPF >>> > > >> > test child finished with -1 >>> > > >> > ---- end ---- >>> > > >> > Test BPF filter subtest 0: FAILED! >>> > > >> > >>> > > >> > Looks like both 1,2 are related. >>> > > >> > Since, CONFIG_DEBUG_INFO is not enabled, I manually copied the vmlinux >>> > > >> > to /boot/ folder. >>> > > >> > >>> > > >> > ------- >>> > > >> > Some more info: >>> > > >> > >>> > > >> > Kernel build dir is set to /lib/modules/4.9.20-sc-amd-x86-64/build >>> > > >> > set env: KBUILD_DIR=/lib/modules/4.9.20-sc-amd-x86-64/build >>> > > >> > unset env: KBUILD_OPTS >>> > > >> > include option is set to -nostdinc -isystem >>> > > >> > /usr/lib/gcc/x86_64-linux-gnu/5/include -I./arch/x86/include >>> > > >> > -I./arch/x86/include/generated/uapi -I./arch/x86/include/generated >>> > > >> > -I./include -I./arch/x86/include/uapi -I./include/uapi >>> > > >> > -I./include/generated/uapi -include ./include/linux/kconfig.h >>> > > >> > set env: NR_CPUS=8 >>> > > >> > set env: LINUX_VERSION_CODE=0x40914 >>> > > >> > set env: CLANG_EXEC=/usr/bin/clang >>> > > >> > set env: CLANG_OPTIONS=-xc >>> > > >> > set env: KERNEL_INC_OPTIONS= -nostdinc -isystem >>> > > >> > /usr/lib/gcc/x86_64-linux-gnu/5/include -I./arch/x86/include >>> > > >> > -I./arch/x86/include/generated/uapi -I./arch/x86/include/generated >>> > > >> > -I./include -I./arch/x86/include/uapi -I./include/uapi >>> > > >> > -I./include/generated/uapi -include ./include/linux/kconfig.h >>> > > >> > set env: WORKING_DIR=/lib/modules/4.9.20-sc-amd-x86-64/build >>> > > >> > >>> > > >> > >>> > > >> > If you have any clue about these failure please help me. >>> > > >> > >>> > > >> > >>> > > >> > Thanks, >>> > > >> > Pintu >>> > > >> > >>> > > >> > >>> > > >> > On Wed, Jan 24, 2018 at 8:23 PM, Pintu Kumar <pintu.ping@xxxxxxxxx> wrote: >>> > > >> >> Hi, >>> > > >> >> >>> > > >> >> Thanks for your help. >>> > > >> >> Yes it was a sub version issue. >>> > > >> >> >>> > > >> >> Earlier, while building the kernel I left the SUBLEVEL as blank. >>> > > >> >> >>> > > >> >> make -j8 bindeb-pkg SUBLEVEL=20 >>> > > >> >> >>> > > >> >> After passing the correct sublevel now the test is working. >>> > > >> >> >>> > > >> >> But still following are failing: >>> > > >> >> >>> > > >> >> 16: Try 'import perf' in python, checking link problems : FAILED! >>> > > >> >> 37.2: Test BPF prologue generation : FAILED! >>> > > >> >> >>> > > >> >> >>> > > >> >> This is the error I get: >>> > > >> >> >>> > > >> >> 16: Try 'import perf' in python, checking link problems : >>> > > >> >> --- start --- >>> > > >> >> test child forked, pid 7637 >>> > > >> >> Traceback (most recent call last): >>> > > >> >> File "<stdin>", line 1, in <module> >>> > > >> >> ImportError: No module named perf >>> > > >> >> test child finished with -1 >>> > > >> >> ---- end ---- >>> > > >> >> Try 'import perf' in python, checking link problems: FAILED! >>> > > >> >> >>> > > >> >> >>> > > >> >> Looking at the vmlinux_path (8 entries long) >>> > > >> >> symsrc__init: cannot get elf header. >>> > > >> >> Failed to find the path for kernel: Invalid ELF file >>> > > >> >> bpf_probe: failed to convert perf probe eventsFailed to add events >>> > > >> >> selected by BPF >>> > > >> >> test child finished with -1 >>> > > >> >> ---- end ---- >>> > > >> >> Test BPF filter subtest 1: FAILED! >>> > > >> >> >>> > > >> >> >>> > > >> >> >>> > > >> >> Thanks, >>> > > >> >> Pintu >>> > > >> >> >>> > > >> >> >>> > > >> >> On Wed, Jan 24, 2018 at 6:39 AM, Wangnan (F) <wangnan0@xxxxxxxxxx> wrote: >>> > > >> >>> >>> > > >> >>> >>> > > >> >>> On 2018/1/23 20:37, Pintu Kumar wrote: >>> > > >> >>>> >>> > > >> >>>> Hi All, >>> > > >> >>>> >>> > > >> >>>> I am verifying all perf tests on Ubuntu-16 x86-64 platform using the >>> > > >> >>>> kernel version 4.9.20. >>> > > >> >>>> I have installed several others packages including: clang, llvm >>> > > >> >>>> >>> > > >> >>>> But, when I run 'perf test' I get some FAILURE. >>> > > >> >>>> Specially, 'perf test LLVM' is failing. >>> > > >> >>>> >>> > > >> >>>> Please check the below error logs: >>> > > >> >>>> >>> > > >> >>>> ---------------------------------------------------- >>> > > >> >>>> # perf test LLVM >>> > > >> >>>> 35: Test LLVM searching and compiling : >>> > > >> >>>> 35.1: Basic BPF llvm compiling test : FAILED! >>> > > >> >>>> 35.2: Test kbuild searching : Skip >>> > > >> >>>> 35.3: Compile source for BPF prologue generation test : Skip >>> > > >> >>>> 35.4: Compile source for BPF relocation test : Skip >>> > > >> >>>> ---------------------------------------------------- >>> > > >> >>>> >>> > > >> >>>> When I run with -v I get this: >>> > > >> >>>> >>> > > >> >>>> ----------------------------------------------------- >>> > > >> >>>> # perf test -v LLVM >>> > > >> >>>> 35: Test LLVM searching and compiling : >>> > > >> >>>> 35.1: Basic BPF llvm compiling test : >>> > > >> >>>> --- start --- >>> > > >> >>>> test child forked, pid 3304 >>> > > >> >>>> Unablt to get kernel version from uname '4.9-xxxx-amd-x86-64' >>> > > >> >>>> WARNING: unable to get correct kernel building directory. >>> > > >> >>>> Hint: Set correct kbuild directory using 'kbuild-dir' option in [llvm] >>> > > >> >>>> section of ~/.perfconfig or set it to "" to suppress kbuild >>> > > >> >>>> detection. >>> > > >> >>>> >>> > > >> >>>> Unablt to get kernel version from uname '4.9-xxxx-amd-x86-64' >>> > > >> >>>> >>> > > >> >>>> >>> > > >> >>>> int _version SEC("version") = LINUX_VERSION_CODE; >>> > > >> >>>> ' | $CLANG_EXEC -D__KERNEL__ -D__NR_CPUS__=$NR_CPUS >>> > > >> >>>> -DLINUX_VERSION_CODE=$LINUX_VERSION_CODE $CLANG_OPTIONS >>> > > >> >>>> $KERNEL_INC_OPTIONS -Wno-unused-value -Wno-pointer-sign >>> > > >> >>>> -working-directory $WORKING_DIR -c "$CLANG_SOURCE" -target bpf -O2 -o >>> > > >> >>>> - >>> > > >> >>>> libbpf: 2129190-4a0 doesn't provide kernel version >>> > > >> >>>> Failed to parse test case 'Basic BPF llvm compiling test' >>> > > >> >>>> test child finished with -1 >>> > > >> >>>> ---- end ---- >>> > > >> >>>> Test LLVM searching and compiling subtest 0: FAILED! >>> > > >> >>>> ----------------------------------------------------- >>> > > >> >>>> >>> > > >> >>>> >>> > > >> >>>> Basic BPF LLVM compiling test is failing. >>> > > >> >>>> It reports that bpf could not able to read the kernel version from uname. >>> > > >> >>>> Is it because of this that 'perf test LLVM' is failing ? >>> > > >> >>>> >>> > > >> >>>> My uname says: >>> > > >> >>>> 4.9-xxxx-amd-x86-64 >>> > > >> >>> >>> > > >> >>> >>> > > >> >>> 'perf test LLVM' requires a well formed uname string (%d.%d.%d). Please see >>> > > >> >>> tools/perf/fetch_kernel_version . If your distro uses another method to >>> > > >> >>> report >>> > > >> >>> kernel version number (from your uname string, sublevel can't be >>> > > >> >>> determined), >>> > > >> >>> please help us improve that function like what we have done for ubuntu >>> > > >> >>> (checking /proc/version_signature). >>> > > >> >>> >>> > > >> >>> >>> > > >> >>> >>> > > >> >>>> My kernel version is: 4.9.20 (from Makefile) >>> > > >> >>>> >>> > > >> >>>> So, I dont think there is any problem with kernel version. >>> > > >> >>>> >>> > > >> >>>> If any body have come across this problem please let me know how to >>> > > >> >>>> resolve this issue. >>> > > >> >>>> >>> > > >> >>>> >>> > > >> >>>> >>> > > >> >>>> Thank You! >>> > > >> >>>> >>> > > >> >>>> Regards, >>> > > >> >>>> Pintu >>> > > >> >>> >>> > > >> >>> >>> > > >> >>> >>> > > > >>> > > > >>> > > > -- >>> > > > Masami Hiramatsu <mhiramat@xxxxxxxxxx> >>> > >>> > >>> > -- >>> > Masami Hiramatsu <mhiramat@xxxxxxxxxx> >> >> >> -- >> Masami Hiramatsu <mhiramat@xxxxxxxxxx> _______________________________________________ Kernelnewbies mailing list Kernelnewbies@xxxxxxxxxxxxxxxxx https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies