On Mon, Apr 5, 2021 at 8:56 PM Yonghong Song <yhs@xxxxxx> wrote:
>
>
>
> On 4/5/21 11:32 AM, Sedat Dilek wrote:
> > On Mon, Apr 5, 2021 at 6:17 PM Yonghong Song <yhs@xxxxxx> wrote:
> >>
> >>
> >>
> >> On 4/4/21 11:55 PM, Sedat Dilek wrote:
> >>> On Mon, Apr 5, 2021 at 4:24 AM Yonghong Song <yhs@xxxxxx> wrote:
> >>>>
> >>>>
> >>>>
> >>>> On 4/4/21 10:25 AM, Sedat Dilek wrote:
> >>>>> On Sun, Apr 4, 2021 at 6:40 PM Yonghong Song <yhs@xxxxxx> wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On 4/4/21 5:46 AM, Sedat Dilek wrote:
> >>>> [...]
> >>>>>>> Next build-error:
> >>>>>>>
> >>>>>>> g++ -g -rdynamic -Wall -O2 -DHAVE_GENHDR
> >>>>>>> -I/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf
> >>>>>>> -I/home/dileks/src/linux-kernel/git/tools/testing/selftests/b
> >>>>>>> pf/tools/include -I/home/dileks/src/linux-kernel/git/include/generated
> >>>>>>> -I/home/dileks/src/linux-kernel/git/tools/lib
> >>>>>>> -I/home/dileks/src/linux-kernel/git/tools/include
> >>>>>>> -I/home/dileks/src/linux-kernel/git/tools/include/uapi
> >>>>>>> -I/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf
> >>>>>>> -Dbpf_prog_load=bpf_prog_test_load
> >>>>>>> -Dbpf_load_program=bpf_test_load_program test_cpp.cpp
> >>>>>>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/test_core_extern.skel.h
> >>>>>>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/tools/build/libbpf/libbpf.a
> >>>>>>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/test_stub.o
> >>>>>>> -lcap -lelf -lz -lrt -lpthread -o
> >>>>>>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/test_cpp
> >>>>>>> /usr/bin/ld: /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/tools/build/libbpf/libbpf.a(libbpf-in.o):
> >>>>>>> relocation R_X86_64_32 against `.rodata.str1.1' ca
> >>>>>>> n not be used when making a PIE object; recompile with -fPIE
> >>>>>>> collect2: error: ld returned 1 exit status
> >>>>>>> make: *** [Makefile:455:
> >>>>>>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/test_cpp]
> >>>>>>> Error 1
> >>>>>>> make: Leaving directory
> >>>>>>> '/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf'
> >>>>>>>
> >>>>>>> LOL, I was not aware that there is usage of *** CXX*** in tools
> >>>>>>> directory (see g++ line and /usr/bin/ld ?).
> >>>>>>>
> >>>>>>> So, I changed my $MAKE_OPTS to use "CXX=clang++".
> >>>>>>
> >>>>>> In kernel, if LLVM=1 is set, we have:
> >>>>>>
> >>>>>> ifneq ($(LLVM),)
> >>>>>> HOSTCC = clang
> >>>>>> HOSTCXX = clang++
> >>>>>> else
> >>>>>> HOSTCC = gcc
> >>>>>> HOSTCXX = g++
> >>>>>> endif
> >>>>>>
> >>>>>> ifneq ($(LLVM),)
> >>>>>> CC = clang
> >>>>>> LD = ld.lld
> >>>>>> AR = llvm-ar
> >>>>>> NM = llvm-nm
> >>>>>> OBJCOPY = llvm-objcopy
> >>>>>> OBJDUMP = llvm-objdump
> >>>>>> READELF = llvm-readelf
> >>>>>> STRIP = llvm-strip
> >>>>>> else
> >>>>>> CC = $(CROSS_COMPILE)gcc
> >>>>>> LD = $(CROSS_COMPILE)ld
> >>>>>> AR = $(CROSS_COMPILE)ar
> >>>>>> NM = $(CROSS_COMPILE)nm
> >>>>>> OBJCOPY = $(CROSS_COMPILE)objcopy
> >>>>>> OBJDUMP = $(CROSS_COMPILE)objdump
> >>>>>> READELF = $(CROSS_COMPILE)readelf
> >>>>>> STRIP = $(CROSS_COMPILE)strip
> >>>>>> endif
> >>>>>>
> >>>>>> So if you have right path, you don't need to set HOSTCC and HOSTCXX
> >>>>>> explicitly.
> >>>>>>
> >>>>>
> >>>>> That is all correct with HOSTCXX but there is no CXX=... assignment
> >>>>> otherwise test_cpp will use g++ as demonstrated.
> >>>>
> >>>> This is not a kernel Makefile issue.
> >>>>
> >>>> We have:
> >>>> testing/selftests/bpf/Makefile:CXX ?= $(CROSS_COMPILE)g++
> >>>>
> >>>> So you need to explicit add CXX=clang++ when compiling
> >>>> bpf selftests with LLVM=1 LLVM_IAS=1.
> >>>>
> >>>
> >>> NOPE.
> >>>
> >>> $ echo $MAKE $MAKE_OPTS
> >>> make V=1 LLVM=1 LLVM_IAS=1 CXX=clang++ PAHOLE=/opt/pahole/bin/pahole
> >>>
> >>> $ LC_ALL=C $MAKE $MAKE_OPTS -C tools/testing/selftests/bpf/ 2>&1 | tee
> >>> ../make-log_tools-testing-selftests-bpf_llvm-1-llvm_ias-1_cxx-clang.txt
> >>>
> >>> This breaks again like reported before:
> >>>
> >>> clang++ -g -rdynamic -Wall -O2 -DHAVE_GENHDR
> >>> -I/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf
> >>> -I/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/tools/include
> >>> -I/home/dileks/src/linux-kernel/git/include/generated
> >>> -I/home/dileks/src/linux-kernel/git/tools/lib
> >>> -I/home/dileks/src/linux-kernel/git/tools/include
> >>> -I/home/dileks/src/linux-kernel/git/tools/include/uapi
> >>> -I/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf
> >>> -Dbpf_prog_load=bpf_prog_test_load
> >>> -Dbpf_load_program=bpf_test_load_program test_cpp.cpp
> >>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/test_core_extern.skel.h
> >>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/tools/build/libbpf/libbpf.a
> >>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/test_stub.o
> >>> -lcap -lelf -lz -lrt -lpthread -o
> >>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/test_cpp
> >>>
> >>> clang-12: warning: treating 'c-header' input as 'c++-header' when in
> >>> C++ mode, this behavior is deprecated [-Wdeprecated]
> >>> clang-12: error: cannot specify -o when generating multiple output files
> >>> make: *** [Makefile:455:
> >>> /home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf/test_cpp]
> >>> Error 1
> >>> make: Leaving directory
> >>> '/home/dileks/src/linux-kernel/git/tools/testing/selftests/bpf'
> >>>
> >>> Do you know some magic CXX flags to be passed?
> >>
> >> I tested in my environment. The reason is LC_ALL=C.
> >> Without LC_ALL=C, make succeeded and with it, test_cpp
> >> compilation failed. Is it possible for you to drop
> >> LC_ALL=C for bpf selftests?
> >>
> >> The following command succeeded for me:
> >> make -C tools/testing/selftests/bpf -j60 LLVM=1 V=1 CXX=clang++ CC=clang
> >>
> >
> > First, I tried the exact make invocation ^^^ in my build-environment
> > but that breaks with the same ERROR.
> >
> > I did in a second run:
> >
> > LLVM_TOOLCHAIN_PATH="/opt/llvm-toolchain/bin"
> > if [ -d ${LLVM_TOOLCHAIN_PATH} ]; then
> > export PATH="${LLVM_TOOLCHAIN_PATH}:${PATH}"
> > fi
> >
> > echo $PATH
> > /opt/llvm-toolchain/bin:/opt/proxychains-ng/bin:/home/dileks/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
> >
> > MAKE="make"
> > MAKE_OPTS="V=1 -j4 LLVM=1 CC=clang CXX=clang++"
> > MAKE_OPTS="$MAKE_OPTS PAHOLE=/opt/pahole/bin/pahole"
> >
> > echo $MAKE $MAKE_OPTS
> > make V=1 -j4 LLVM=1 CC=clang CXX=clang++ PAHOLE=/opt/pahole/bin/pahole
> >
> > $MAKE $MAKE_OPTS -C tools/testing/selftests/bpf/ 2>&1 | tee
> > ../make-log_tools-testing-selftests-bpf.txt
> >
> > That would have been funny... Drop LC_ALL=C from make line as a fix.
> >
> > Just curious: Do you see these warnings?
> >
> > clang-12: warning: argument unused during compilation: '-rdynamic'
> > [-Wunused-command-line-argument]
> > clang-12: warning: -lcap: 'linker' input unused [-Wunused-command-line-argument]
> > clang-12: warning: -lelf: 'linker' input unused [-Wunused-command-line-argument]
> > clang-12: warning: -lz: 'linker' input unused [-Wunused-command-line-argument]
> > clang-12: warning: -lrt: 'linker' input unused [-Wunused-command-line-argument]
> > clang-12: warning: -lpthread: 'linker' input unused
> > [-Wunused-command-line-argument]
> > clang-12: warning: -lm: 'linker' input unused [-Wunused-command-line-argument]
> >
> > Equivalent CFLAGS for '-rdynamic' when CC=clang is used?
> > Missing LDFLAGS when LD=ld.lld (make LLVM=1) is used?
>
> I see this warning as well, but seems it does not hurt...
> Maybe some flags need change if the CC is clang...
>
> >
> > Last question:
> > Can you pass LLVM_IAS=1 (means use LLVM/Clang Integrated ASsembler) to
> > your make line?
> >
> > Old: make -C tools/testing/selftests/bpf -j60 LLVM=1 V=1 CXX=clang++ CC=clang
> > New: make -C tools/testing/selftests/bpf -j60 LLVM=1 LLVM_IAS=1 V=1
> > CXX=clang++ CC=clang
> >
> > Does it build successfully?
>
> I think it is better to add LLVM_IAS=1. In my build, for non-lto, I
> didn't pass LLVM_IAS=1 to kernel build, so selftest does not need it either.
>
> But for LTO build, LLVM_IAS=1 is required so selftest also needs
> LLVM_IAS=1.
>
> Yes, we can always have LLVM_IAS=1 if it is included in kernel build.
>
For Clang-LTO builds LLVM_IAS=1 is mandatory.
- Sedat -
> >
> > - Sedat -
> >
> >>>
> >>> The only solution is to suppress the build of test_cpp (see
> >>> TEST_GEN_PROGS_EXTENDED):
> >>>
> >>> $ git diff tools/testing/selftests/bpf/Makefile
> >>> diff --git a/tools/testing/selftests/bpf/Makefile
> >>> b/tools/testing/selftests/bpf/Makefile
> >>> index 044bfdcf5b74..cf7c7c8f72cf 100644
> >>> --- a/tools/testing/selftests/bpf/Makefile
> >>> +++ b/tools/testing/selftests/bpf/Makefile
> >>> @@ -77,8 +77,8 @@ TEST_PROGS_EXTENDED := with_addr.sh \
> >>> # Compile but not part of 'make run_tests'
> >>> TEST_GEN_PROGS_EXTENDED = test_sock_addr test_skb_cgroup_id_user \
> >>> flow_dissector_load test_flow_dissector test_tcp_check_syncookie_user \
> >>> - test_lirc_mode2_user xdping test_cpp runqslower bench bpf_testmod.ko \
> >>> - xdpxceiver
> >>> + test_lirc_mode2_user xdping runqslower bench bpf_testmod.ko xdpxceiver
> >>> + # test_cpp # Suppress the build when CXX=clang++ is used
> >>>
> >>> TEST_CUSTOM_PROGS = $(OUTPUT)/urandom_read
> >>>
> >>> I have attached both make-logs with and without suppressing the build
> >>> of test_cpp and the diff.
> >>>
> >>> - Sedat -
> >>>
> >>>>
> >>>>>
> >> [...]
