From: Martin KaFai Lau <martin.lau@xxxxxxxxxx> This set is to use bpf_mem_cache_alloc/free in bpf_local_storage. The primary motivation is to solve the deadlock/recursion issue when bpf_task_storage is used in a bpf tracing prog [1]. This set also comes with a micro-benchmark to test the storage creation. Patch 1 to 4 are some general cleanup after bpf_local_storage has been extended to multiple kernel objects (sk, task, inode, and then cgrp). Patch 5 to 11 is to refactor the memory free logic into the new bpf_selem_free() and bpf_local_storage_free() functions. Together with the existing bpf_selem_alloc() and bpf_local_storage_alloc(), it should provide an easier way to change the alloc/free path in the future. Patch 12 to 14 is to use bpf_mem_cache_alloc/free. The remaining patches are selftests and benchmark. [1]: https://lore.kernel.org/bpf/20221118190109.1512674-1-namhyung@xxxxxxxxxx/ v2: - Added bpf_mem_cache_alloc_flags() and bpf_mem_cache_raw_free() to hide the internal data structure of the bpf allocator. - Fixed a typo bug in bpf_selem_free() - Simplified the test_local_storage test by directly using err returned from libbpf Martin KaFai Lau (17): bpf: Move a few bpf_local_storage functions to static scope bpf: Refactor codes into bpf_local_storage_destroy bpf: Remove __bpf_local_storage_map_alloc bpf: Remove the preceding __ from __bpf_selem_unlink_storage bpf: Remember smap in bpf_local_storage bpf: Repurpose use_trace_rcu to reuse_now in bpf_local_storage bpf: Remove bpf_selem_free_fields*_rcu bpf: Add bpf_selem_free_rcu callback bpf: Add bpf_selem_free() bpf: Add bpf_local_storage_rcu callback bpf: Add bpf_local_storage_free() bpf: Add a few bpf mem allocator functions bpf: Use bpf_mem_cache_alloc/free in bpf_selem_alloc/free bpf: Use bpf_mem_cache_alloc/free for bpf_local_storage selftests/bpf: Replace CHECK with ASSERT in test_local_storage selftests/bpf: Check freeing sk->sk_local_storage with sk_local_storage->smap is NULL selftests/bpf: Add local-storage-create benchmark include/linux/bpf_local_storage.h | 21 +- include/linux/bpf_mem_alloc.h | 2 + kernel/bpf/bpf_cgrp_storage.c | 11 +- kernel/bpf/bpf_inode_storage.c | 10 +- kernel/bpf/bpf_local_storage.c | 279 ++++++++++-------- kernel/bpf/bpf_task_storage.c | 11 +- kernel/bpf/memalloc.c | 42 ++- net/core/bpf_sk_storage.c | 12 +- tools/testing/selftests/bpf/Makefile | 2 + tools/testing/selftests/bpf/bench.c | 2 + .../bpf/benchs/bench_local_storage_create.c | 141 +++++++++ .../bpf/prog_tests/test_local_storage.c | 47 ++- .../bpf/progs/bench_local_storage_create.c | 57 ++++ .../selftests/bpf/progs/local_storage.c | 29 +- 14 files changed, 446 insertions(+), 220 deletions(-) create mode 100644 tools/testing/selftests/bpf/benchs/bench_local_storage_create.c create mode 100644 tools/testing/selftests/bpf/progs/bench_local_storage_create.c -- 2.34.1
