On 10/5/20 12:27 PM, Andrii Nakryiko wrote:
On Fri, Oct 2, 2020 at 10:16 PM Yonghong Song <yhs@xxxxxx> wrote:
On 10/2/20 9:22 PM, Andrii Nakryiko wrote:
On Fri, Oct 2, 2020 at 7:19 PM Yonghong Song <yhs@xxxxxx> wrote:
With latest llvm trunk, bpf programs under samples/bpf
directory, if using CORE, may experience the following
errors:
LLVM ERROR: Cannot select: intrinsic %llvm.preserve.struct.access.index
PLEASE submit a bug report to https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.llvm.org_&d=DwIBaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=DA8e1B5r073vIqRrFz7MRA&m=_D9tvuWQ6EbYqcMBdVB0qqRMVdV6Etws5ITtx8Pa1ZM&s=BwTAvhipPl-Az_WaiJDbqU8yl__NvG8W4HmCqWqHdqg&e= and include the crash backtrace.
Stack dump:
0. Program arguments: llc -march=bpf -filetype=obj -o samples/bpf/test_probe_write_user_kern.o
1. Running pass 'Function Pass Manager' on module '<stdin>'.
2. Running pass 'BPF DAG->DAG Pattern Instruction Selection' on function '@bpf_prog1'
#0 0x000000000183c26c llvm::sys::PrintStackTrace(llvm::raw_ostream&, int)
(/data/users/yhs/work/llvm-project/llvm/build.cur/install/bin/llc+0x183c26c)
...
#7 0x00000000017c375e (/data/users/yhs/work/llvm-project/llvm/build.cur/install/bin/llc+0x17c375e)
#8 0x00000000016a75c5 llvm::SelectionDAGISel::CannotYetSelect(llvm::SDNode*)
(/data/users/yhs/work/llvm-project/llvm/build.cur/install/bin/llc+0x16a75c5)
#9 0x00000000016ab4f8 llvm::SelectionDAGISel::SelectCodeCommon(llvm::SDNode*, unsigned char const*,
unsigned int) (/data/users/yhs/work/llvm-project/llvm/build.cur/install/bin/llc+0x16ab4f8)
...
Aborted (core dumped) | llc -march=bpf -filetype=obj -o samples/bpf/test_probe_write_user_kern.o
The reason is due to llvm change https://urldefense.proofpoint.com/v2/url?u=https-3A__reviews.llvm.org_D87153&d=DwIBaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=DA8e1B5r073vIqRrFz7MRA&m=_D9tvuWQ6EbYqcMBdVB0qqRMVdV6Etws5ITtx8Pa1ZM&s=fo_LvXqHJx_m0m0pJJiDdOUcOzVXm2_iYoxPhpqpzng&e=
where the CORE relocation global generation is moved from the beginning
of target dependent optimization (llc) to the beginning
of target independent optimization (opt).
Since samples/bpf programs did not use vmlinux.h and its clang compilation
uses native architecture, we need to adjust arch triple at opt level
to do CORE relocation global generation properly. Otherwise, the above
error will appear.
This patch fixed the issue by introduce opt and llvm-dis to compilation chain,
which will do proper CORE relocation global generation as well as O2 level
optimization. Tested with llvm10, llvm11 and trunk/llvm12.
Signed-off-by: Yonghong Song <yhs@xxxxxx>
---
samples/bpf/Makefile | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile
index 4f1ed0e3cf9f..79c5fdea63d2 100644
--- a/samples/bpf/Makefile
+++ b/samples/bpf/Makefile
@@ -211,6 +211,8 @@ TPROGLDLIBS_xsk_fwd += -pthread
# make M=samples/bpf/ LLC=~/git/llvm/build/bin/llc CLANG=~/git/llvm/build/bin/clang
LLC ?= llc
CLANG ?= clang
+OPT ?= opt
+LLVM_DIS ?= llvm-dis
LLVM_OBJCOPY ?= llvm-objcopy
BTF_PAHOLE ?= pahole
@@ -314,7 +316,9 @@ $(obj)/%.o: $(src)/%.c
-Wno-address-of-packed-member -Wno-tautological-compare \
-Wno-unknown-warning-option $(CLANG_ARCH_ARGS) \
-I$(srctree)/samples/bpf/ -include asm_goto_workaround.h \
- -O2 -emit-llvm -c $< -o -| $(LLC) -march=bpf $(LLC_FLAGS) -filetype=obj -o $@
+ -O2 -emit-llvm -Xclang -disable-llvm-passes -c $< -o - | \
+ $(OPT) -O2 -mtriple=bpf-pc-linux | $(LLVM_DIS) | \
+ $(LLC) -march=bpf $(LLC_FLAGS) -filetype=obj -o $@
This is an extremely unusual set of steps, and might be worthwhile to
leave a comment explaining what's going on, so that I or someone else
doesn't ask the same question few months later :)
Sure. Will add some comments and send v2.
At any rate, this fixes the issue, so:
Acked-by: Andrii Nakryiko <andrii@xxxxxxxxxx>
I keep forgetting exact details of why we do this native clang + llc
pipeline instead of just doing `clang -target bpf`? Is it still
samples/bpf programs did not use vmlinux.h. they directly use
kernel-devel headers, hence they need to first compile with native arch
for clang but later change target arch to bpf to generate final byte code.
They cannot just do 'clang -target bpf' without vmlinux.h.
Ok, right, thanks for explanation. I wonder if this "native" clang +
llc pass will also help with vmlinux.h on 32-bit architectures (though
with my recent patches that shouldn't be necessary, so this is more of
a curiosity, rather than the need).
Yes, I think this should help 32-bit as all the 32-bit long/ptr size is
captured in IR. I probably tried before with 32bit x86 uprobe, but I
forgot details...
But changing to use vmlinux.h is a much bigger project and I merely
want to make it just work so people won't make/compile samples/bpf
and get compilation errors.
Right, of course.
relevant and necessary, or we can just simplify it now?
ifeq ($(DWARF2BTF),y)
$(BTF_PAHOLE) -J $@
endif
--
2.24.1
