On 5/25/21 11:29 AM, Fangrui Song wrote:
I have a review queue with a huge pile of LLVM patches and have only
skimmed through this.
First, if the size benefit of REL over RELA isn't deem that necessary,
I will highly recommend RELA for simplicity and robustness.
REL is error-prone.
The worry is backward compatibility. Because of BPF ELF format
is so intervened with bpf eco system (loading, bpf map, etc.),
a lot of tools in the wild already implemented to parse REL...
It will be difficult to change...
On 2021-05-24, Yonghong Song wrote:
LLVM upstream commit
https://reviews.llvm.org/D102712
made some changes to bpf relocations to make them
llvm linker lld friendly. The scope of
existing relocations R_BPF_64_{64,32} is narrowed
and new relocations R_BPF_64_{ABS32,ABS64,NODYLD32}
are introduced.
Let us add some documentation about llvm bpf
relocations so people can understand how to resolve
them properly in their respective tools.
Cc: John Fastabend <john.fastabend@xxxxxxxxx>
Cc: Lorenz Bauer <lmb@xxxxxxxxxxxxxx>
Signed-off-by: Yonghong Song <yhs@xxxxxx>
---
Documentation/bpf/index.rst | 1 +
Documentation/bpf/llvm_reloc.rst | 240 +++++++++++++++++++++++++
tools/testing/selftests/bpf/README.rst | 19 ++
3 files changed, 260 insertions(+)
create mode 100644 Documentation/bpf/llvm_reloc.rst
Changelogs:
v1 -> v2:
- add an example to illustrate how relocations related to base
section and symbol table and what is "Implicit Addend"
- clarify why we use 32bit read/write for R_BPF_64_64 (ld_imm64)
relocations.
diff --git a/Documentation/bpf/index.rst b/Documentation/bpf/index.rst
index a702f67dd45f..93e8cf12a6d4 100644
--- a/Documentation/bpf/index.rst
+++ b/Documentation/bpf/index.rst
@@ -84,6 +84,7 @@ Other
:maxdepth: 1
ringbuf
+ llvm_reloc
.. Links:
.. _networking-filter: ../networking/filter.rst
diff --git a/Documentation/bpf/llvm_reloc.rst
b/Documentation/bpf/llvm_reloc.rst
new file mode 100644
index 000000000000..5ade0244958f
--- /dev/null
+++ b/Documentation/bpf/llvm_reloc.rst
@@ -0,0 +1,240 @@
+.. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+
+====================
+BPF LLVM Relocations
+====================
+
+This document describes LLVM BPF backend relocation types.
+
+Relocation Record
+=================
+
+LLVM BPF backend records each relocation with the following 16-byte
+ELF structure::
+
+ typedef struct
+ {
+ Elf64_Addr r_offset; // Offset from the beginning of section.
+ Elf64_Xword r_info; // Relocation type and symbol index.
+ } Elf64_Rel;
+
+For example, for the following code::
+
+ int g1 __attribute__((section("sec")));
+ int g2 __attribute__((section("sec")));
+ static volatile int l1 __attribute__((section("sec")));
+ static volatile int l2 __attribute__((section("sec")));
+ int test() {
+ return g1 + g2 + l1 + l2;
+ }
+
+Compiled with ``clang -target bpf -O2 -c test.c``, the following is
+the code with ``llvm-objdump -dr test.o``::
+
+ 0: 18 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r1 =
0 ll
+ 0000000000000000: R_BPF_64_64 g1
+ 2: 61 11 00 00 00 00 00 00 r1 = *(u32 *)(r1 + 0)
+ 3: 18 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r2 =
0 ll
+ 0000000000000018: R_BPF_64_64 g2
+ 5: 61 20 00 00 00 00 00 00 r0 = *(u32 *)(r2 + 0)
+ 6: 0f 10 00 00 00 00 00 00 r0 += r1
+ 7: 18 01 00 00 08 00 00 00 00 00 00 00 00 00 00 00 r1 =
8 ll
+ 0000000000000038: R_BPF_64_64 sec
+ 9: 61 11 00 00 00 00 00 00 r1 = *(u32 *)(r1 + 0)
+ 10: 0f 10 00 00 00 00 00 00 r0 += r1
+ 11: 18 01 00 00 0c 00 00 00 00 00 00 00 00 00 00 00 r1 =
12 ll
+ 0000000000000058: R_BPF_64_64 sec
+ 13: 61 11 00 00 00 00 00 00 r1 = *(u32 *)(r1 + 0)
+ 14: 0f 10 00 00 00 00 00 00 r0 += r1
+ 15: 95 00 00 00 00 00 00 00 exit
+
+There are four relations in the above for four ``LD_imm64``
instructions.
+The following ``llvm-readelf -r test.o`` shows the binary values of
the four
+relocations::
+
+ Relocation section '.rel.text' at offset 0x190 contains 4 entries:
+ Offset Info Type Symbol's
Value Symbol's Name
+ 0000000000000000 0000000600000001 R_BPF_64_64
0000000000000000 g1
+ 0000000000000018 0000000700000001 R_BPF_64_64
0000000000000004 g2
+ 0000000000000038 0000000400000001 R_BPF_64_64
0000000000000000 sec
+ 0000000000000058 0000000400000001 R_BPF_64_64
0000000000000000 sec
+
+Each relocation is represented by ``Offset`` (8 bytes) and ``Info``
(8 bytes).
+For example, the first relocation corresponds to the first instruction
+(Offset 0x0) and the corresponding ``Info`` indicates the relocation
type
+of ``R_BPF_64_64`` (type 1) and the entry in the symbol table (entry 6).
+The following is the symbol table with ``llvm-readelf -s test.o``::
+
+ Symbol table '.symtab' contains 8 entries:
+ Num: Value Size Type Bind Vis Ndx Name
+ 0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
+ 1: 0000000000000000 0 FILE LOCAL DEFAULT ABS test.c
+ 2: 0000000000000008 4 OBJECT LOCAL DEFAULT 4 l1
+ 3: 000000000000000c 4 OBJECT LOCAL DEFAULT 4 l2
+ 4: 0000000000000000 0 SECTION LOCAL DEFAULT 4 sec
+ 5: 0000000000000000 128 FUNC GLOBAL DEFAULT 2 test
+ 6: 0000000000000000 4 OBJECT GLOBAL DEFAULT 4 g1
+ 7: 0000000000000004 4 OBJECT GLOBAL DEFAULT 4 g2
+
+The 6th entry is global variable ``g1`` with value 0.
+
+Similarly, the second relocation is at ``.text`` offset ``0x18``,
instruction 3,
+for global variable ``g2`` which has a symbol value 4, the offset
+from the start of ``.data`` section.
+
+The third and fourth relocations refers to static variables ``l1``
+and ``l2``. From ``.rel.text`` section above, it is not clear
+which symbols they really refers to as they both refers to
+symbol table entry 4, symbol ``sec``, which has ``SECTION`` type
STT_SECTION. `SECTION` is just an abbreviated form used by some binary
tools.
This is just to match llvm-readelf output. I can add a reference
to STT_SECTION for the right macro name.
+and represents a section. So for static variable or function,
+the section offset is written to the original insn
+buffer, which is called ``IA`` (implicit addend). Looking at
+above insn ``7`` and ``11``, they have section offset ``8`` and ``12``.
+From symbol table, we can find that they correspond to entries ``2``
+and ``3`` for ``l1`` and ``l2``.
The other REL based psABIs all use `A` for addend.
I can use `A` as well. The reason I am using `IA` since it is not
shown in the relocation record and lld used API 'getImplicitAddend()`
get this value. But I can certainly use `A`.
+In general, the ``IA`` is 0 for global variables and functions,
+and is the section offset or some computation result based on
+section offset for static variables/functions. The non-section-offset
+case refers to function calls. See below for more details.
+
+Different Relocation Types
+==========================
+
+Six relocation types are supported. The following is an overview and
+``S`` represents the value of the symbol in the symbol table::
+
+ Enum ELF Reloc Type Description BitSize Offset
Calculation
+ 0 R_BPF_NONE None
+ 1 R_BPF_64_64 ld_imm64 insn 32 r_offset + 4 S
+ IA
+ 2 R_BPF_64_ABS64 normal data 64 r_offset S
+ IA
+ 3 R_BPF_64_ABS32 normal data 32 r_offset S
+ IA
+ 4 R_BPF_64_NODYLD32 .BTF[.ext] data 32 r_offset S
+ IA
+ 10 R_BPF_64_32 call insn 32 r_offset + 4 (S
+ IA) / 8 - 1
Shifting the offset by 4 looks weird. R_386_32 applies at r_offset.
The call instruction R_BPF_64_32 is strange. Such special calculation
should not be named R_BPF_64_32.
Again, we have a backward compatibility issue here. I would like to
provide an alias for it in llvm relocation header file, but I don't
know how to do it.
+For example, ``R_BPF_64_64`` relocation type is used for ``ld_imm64``
instruction.
+The actual to-be-relocated data (0 or section offset)
+is stored at ``r_offset + 4`` and the read/write
+data bitsize is 32 (4 bytes). The relocation can be resolved with
+the symbol value plus implicit addend. Note that the ``BitSize`` is
32 which
+means the section offset must be less than or equal to ``UINT32_MAX``
and this
+is enforced by LLVM BPF backend.
+
+In another case, ``R_BPF_64_ABS64`` relocation type is used for
normal 64-bit data.
+The actual to-be-relocated data is stored at ``r_offset`` and the
read/write data
+bitsize is 64 (8 bytes). The relocation can be resolved with
+the symbol value plus implicit addend.
+
+Both ``R_BPF_64_ABS32`` and ``R_BPF_64_NODYLD32`` types are for
32-bit data.
+But ``R_BPF_64_NODYLD32`` specifically refers to relocations in
``.BTF`` and
+``.BTF.ext`` sections. For cases like bcc where llvm
``ExecutionEngine RuntimeDyld``
+is involved, ``R_BPF_64_NODYLD32`` types of relocations should not be
resolved
+to actual function/variable address. Otherwise, ``.BTF`` and
``.BTF.ext``
+become unusable by bcc and kernel.
Why cannot R_BPF_64_ABS32 cover the use cases of R_BPF_64_NODYLD32?
I haven't seen any relocation type which hard coding knowledge on data
sections.
This is due to how .BTF relocation is done. Relocation is against
loadable symbols but it does not want dynamic linker to resolve them.
Instead it wants libbpf and kernel to resolve them in a different
way.
+Type ``R_BPF_64_32`` is used for call instruction. The call target
section
+offset is stored at ``r_offset + 4`` (32bit) and calculated as
+``(S + IA) / 8 - 1``.
In other ABIs, names like 32/ABS32/ABS64 refer to absolute relocation types
without such complex operation.
Again, this is a historical artifact to handle call instruction. I am
aware that this might be different from other architectures. But we have
to keep backward compatibility...
+Examples
+========
+
+Types ``R_BPF_64_64`` and ``R_BPF_64_32`` are used to resolve
``ld_imm64``
+and ``call`` instructions. For example::
+
+ __attribute__((noinline)) __attribute__((section("sec1")))
+ int gfunc(int a, int b) {
+ return a * b;
+ }
+ static __attribute__((noinline)) __attribute__((section("sec1")))
+ int lfunc(int a, int b) {
+ return a + b;
+ }
+ int global __attribute__((section("sec2")));
+ int test(int a, int b) {
+ return gfunc(a, b) + lfunc(a, b) + global;
+ }
+
[...]
