On Thu, Aug 24, 2023 at 4:02 PM Eduard Zingerman <eddyz87@xxxxxxxxx> wrote:
>
> Add a section on CO-RE relocations to llvm_relo.rst.
> Describe relevant .BTF.ext structure, `enum bpf_core_relo_kind`
> and `struct bpf_core_relo` in some detail.
> Description is based on doc-string from include/uapi/linux/bpf.h.
>
> Signed-off-by: Eduard Zingerman <eddyz87@xxxxxxxxx>
> ---
Looks great overall, thanks a lot for adding this!
> Documentation/bpf/btf.rst | 27 ++++-
> Documentation/bpf/llvm_reloc.rst | 178 +++++++++++++++++++++++++++++++
> 2 files changed, 201 insertions(+), 4 deletions(-)
>
> diff --git a/Documentation/bpf/btf.rst b/Documentation/bpf/btf.rst
> index f32db1f44ae9..c0530211c3c1 100644
> --- a/Documentation/bpf/btf.rst
> +++ b/Documentation/bpf/btf.rst
> @@ -726,8 +726,8 @@ same as the one describe in :ref:`BTF_Type_String`.
> 4.2 .BTF.ext section
> --------------------
>
> -The .BTF.ext section encodes func_info and line_info which needs loader
> -manipulation before loading into the kernel.
> +The .BTF.ext section encodes func_info, line_info and CO-RE relocations
> +which needs loader manipulation before loading into the kernel.
>
> The specification for .BTF.ext section is defined at ``tools/lib/bpf/btf.h``
> and ``tools/lib/bpf/btf.c``.
> @@ -745,11 +745,16 @@ The current header of .BTF.ext section::
> __u32 func_info_len;
> __u32 line_info_off;
> __u32 line_info_len;
> +
> + /* optional part of .BTF.ext header */
> + __u32 core_relo_off;
> + __u32 core_relo_len;
> };
>
> It is very similar to .BTF section. Instead of type/string section, it
> -contains func_info and line_info section. See :ref:`BPF_Prog_Load` for details
> -about func_info and line_info record format.
> +contains func_info, line_info and core_relo sub-sections.
> +See :ref:`BPF_Prog_Load` for details about func_info and line_info
> +record format.
>
> The func_info is organized as below.::
>
> @@ -787,6 +792,20 @@ kernel API, the ``insn_off`` is the instruction offset in the unit of ``struct
> bpf_insn``. For ELF API, the ``insn_off`` is the byte offset from the
> beginning of section (``btf_ext_info_sec->sec_name_off``).
>
> +The core_relo is organized as below.::
> +
> + core_relo_rec_size
nit: should we specify that this is __u32 value? Same for func_info
and line_info. I'm not sure we ever explicitly mention this this
record size is 4 byte long.
> + btf_ext_info_sec for section #1 /* core_relo for section #1 */
> + btf_ext_info_sec for section #2 /* core_relo for section #2 */
> +
> +``core_relo_rec_size`` specifies the size of ``bpf_core_relo``
> +structure when .BTF.ext is generated. All ``bpf_core_relo`` structures
> +within a single ``btf_ext_info_sec`` describe relocations applied to
> +section named by ``btf_ext_info_sec::sec_name_off``.
> +
> +See :ref:`Documentation/bpf/llvm_reloc <btf-co-re-relocations>`
> +for more information on CO-RE relocations.
> +
> 4.2 .BTF_ids section
> --------------------
>
> diff --git a/Documentation/bpf/llvm_reloc.rst b/Documentation/bpf/llvm_reloc.rst
> index 450e6403fe3d..efe0b6ea4921 100644
> --- a/Documentation/bpf/llvm_reloc.rst
> +++ b/Documentation/bpf/llvm_reloc.rst
> @@ -240,3 +240,181 @@ The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations::
> Offset Info Type Symbol's Value Symbol's Name
> 000000000000002c 0000000200000004 R_BPF_64_NODYLD32 0000000000000000 .text
> 0000000000000040 0000000200000004 R_BPF_64_NODYLD32 0000000000000000 .text
> +
> +.. _btf-co-re-relocations:
> +
> +=================
> +CO-RE Relocations
> +=================
> +
> +From object file point of view CO-RE mechanism is implemented as a set
> +of CO-RE specific relocation records. These relocation records are not
> +related to ELF relocations and are encoded in .BTF.ext section.
> +See :ref:`Documentation/bpf/btf <BTF_Ext_Section>` for more
> +information on .BTF.ext structure.
> +
> +
> +CO-RE relocations are applied to BPF instructions to update immediate
> +or offset fields of the instruction at load time with information
> +relevant for target kernel.
> +
> +Relocation kinds
> +================
> +
> +There are several kinds of CO-RE relocations that could be split in
> +three groups:
> +
> +* Field-based - patch instruction with field related information, e.g.
> + change offset field of the BPF_LD instruction to reflect offset
> + of a specific structure field in the target kernel.
> +
> +* Type-based - patch instruction with type related information, e.g.
> + change immediate field of the BPF_MOV instruction to 0 or 1 to
> + reflect if specific type is present in the target kernel.
> +
> +* Enum-based - patch instruction with enum related information, e.g.
> + change immediate field of the BPF_MOV instruction to reflect value
> + of a specific enum literal in the target kernel.
> +
Instead of referencing BPF_MOV specifically, would it be useful to
incorporate all the different instructions that can be relocated?
bpf_core_patch_insn comment has a nice summary, maybe we can somehow
reuse it in this doc as well?
* Currently supported classes of BPF instruction are:
* 1. rX = <imm> (assignment with immediate operand);
* 2. rX += <imm> (arithmetic operations with immediate operand);
* 3. rX = <imm64> (load with 64-bit immediate value);
* 4. rX = *(T *)(rY + <off>), where T is one of {u8, u16, u32, u64};
* 5. *(T *)(rX + <off>) = rY, where T is one of {u8, u16, u32, u64};
* 6. *(T *)(rX + <off>) = <imm>, where T is one of {u8, u16, u32, u64}.
> +The complete list of relocation kinds is represented by the following enum:
> +
> +.. code-block:: c
> +
> + enum bpf_core_relo_kind {
> + BPF_CORE_FIELD_BYTE_OFFSET = 0, /* field byte offset */
> + BPF_CORE_FIELD_BYTE_SIZE = 1, /* field size in bytes */
> + BPF_CORE_FIELD_EXISTS = 2, /* field existence in target kernel */
> + BPF_CORE_FIELD_SIGNED = 3, /* field signedness (0 - unsigned, 1 - signed) */
> + BPF_CORE_FIELD_LSHIFT_U64 = 4, /* bitfield-specific left bitshift */
> + BPF_CORE_FIELD_RSHIFT_U64 = 5, /* bitfield-specific right bitshift */
> + BPF_CORE_TYPE_ID_LOCAL = 6, /* type ID in local BPF object */
> + BPF_CORE_TYPE_ID_TARGET = 7, /* type ID in target kernel */
> + BPF_CORE_TYPE_EXISTS = 8, /* type existence in target kernel */
> + BPF_CORE_TYPE_SIZE = 9, /* type size in bytes */
> + BPF_CORE_ENUMVAL_EXISTS = 10, /* enum value existence in target kernel */
> + BPF_CORE_ENUMVAL_VALUE = 11, /* enum value integer value */
> + BPF_CORE_TYPE_MATCHES = 12, /* type match in target kernel */
> + };
> +
> +CO-RE Relocation Record
> +=======================
> +
[...]
> +CO-RE Relocation Examples
> +=========================
> +
> +For the following C code:
> +
> +.. code-block:: c
> +
> + struct foo {
> + int a;
> + int b;
> + } __attribute__((preserve_access_index));
> +
> + enum bar { U, V };
> +
> + void buz(struct foo *s, volatile unsigned long *g) {
> + s->a = 1;
> + *g = __builtin_preserve_field_info(s->b, 1);
> + *g = __builtin_preserve_type_info(*s, 1);
> + *g = __builtin_preserve_enum_value(*(enum bar *)V, 1);
> + }
> +
> +With the following BTF definititions:
Gmail points to typo in "definitions"
> +
> +.. code-block::
> +
> + ...
> + [2] STRUCT 'foo' size=8 vlen=2
> + 'a' type_id=3 bits_offset=0
> + 'b' type_id=3 bits_offset=32
> + [3] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
> + ...
> + [9] ENUM 'bar' encoding=UNSIGNED size=4 vlen=2
> + 'U' val=0
> + 'V' val=1
> +
> +The following relocation entries would be generated:
> +
> +.. code-block:: c
> +
> + <buz>:
> + 0: *(u32 *)(r1 + 0x0) = 0x1
> + 00: CO-RE <byte_off> [2] struct foo::a (0:0)
> + 1: r1 = 0x4
> + 08: CO-RE <byte_sz> [2] struct foo::b (0:1)
> + 2: *(u64 *)(r2 + 0x0) = r1
> + 3: r1 = 0x8
> + 18: CO-RE <type_size> [2] struct foo
> + 4: *(u64 *)(r2 + 0x0) = r1
> + 5: r1 = 0x1 ll
> + 28: CO-RE <enumval_value> [9] enum bar::V = 1
> + 7: *(u64 *)(r2 + 0x0) = r1
> + 8: exit
> +
> +Note: modifications for llvm-objdump to show these relocation entries
> +are currently work in progress.
Do we need this note here? Doesn't seem like you have any other
reference to llvm-objdump?
> --
> 2.41.0
>
