The BPF struct_ops previously only allowed for one page to be used for the trampolines of all links in a map. However, we have recently run out of space due to the large number of BPF program links. By allocating additional pages when we exhaust an existing page, we can accommodate more links in a single map. The variable st_map->image has been changed to st_map->image_pages, and its type has been changed to an array of pointers to buffers of PAGE_SIZE. Additional pages are allocated when all existing pages are exhausted. The test case loads a struct_ops maps having 40 programs. Their trampolines takes about 6.6k+ bytes over 1.5 pages on x86. --- Major differences from v3: - Refactor buffer allocations to bpf_struct_ops_tramp_buf_alloc() and bpf_struct_ops_tramp_buf_free(). Major differences from v2: - Move image buffer allocation to bpf_struct_ops_prepare_trampoline(). Major differences from v1: - Always free pages if failing to update. - Allocate 8 pages at most. v3: https://lore.kernel.org/all/20240224030302.1500343-1-thinker.li@xxxxxxxxx/ v2: https://lore.kernel.org/all/20240221225911.757861-1-thinker.li@xxxxxxxxx/ v1: https://lore.kernel.org/all/20240216182828.201727-1-thinker.li@xxxxxxxxx/ Kui-Feng Lee (3): bpf, net: validate struct_ops when updating value. bpf: struct_ops supports more than one page for trampolines. selftests/bpf: Test struct_ops maps with a large number of program links. include/linux/bpf.h | 4 +- kernel/bpf/bpf_struct_ops.c | 139 ++++++++++++------ net/bpf/bpf_dummy_struct_ops.c | 12 +- net/ipv4/tcp_cong.c | 6 +- .../selftests/bpf/bpf_testmod/bpf_testmod.h | 44 ++++++ .../prog_tests/test_struct_ops_multi_pages.c | 30 ++++ .../bpf/progs/struct_ops_multi_pages.c | 102 +++++++++++++ 7 files changed, 280 insertions(+), 57 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/test_struct_ops_multi_pages.c create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_multi_pages.c -- 2.34.1
