This patchset adds a new interface and use it to select victim when OOM is invoked. The mainly motivation is the need to customizable OOM victim selection functionality. The new interface is a bpf hook plugged in oom_evaluate_task. It takes oc and current task as parameters and return a result indicating which one is selected by the attached bpf program. There are several conserns when designing this interface suggested by Michal: 1. Hooking into oom_evaluate_task can keep the consistency of global and memcg OOM interface. Besides, it seems the least disruptive to the existing oom killer implementation. 2. Userspace can handle a lot on its own and provide the input to the BPF program to make a decision. Since the oom scope iteration will be implemented already in the kernel so all the BPF program has to do is to rank processes or memcgs. Previous discussion link: [1]https://lore.kernel.org/lkml/20230727073632.44983-1-zhouchuyi@xxxxxxxxxxxxx/ Chuyi Zhou (2): mm, oom: Introduce bpf_select_task bpf: Add OOM policy test mm/oom_kill.c | 57 ++++++- .../bpf/prog_tests/test_oom_policy.c | 140 ++++++++++++++++++ .../testing/selftests/bpf/progs/oom_policy.c | 77 ++++++++++ 3 files changed, 267 insertions(+), 7 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/test_oom_policy.c create mode 100644 tools/testing/selftests/bpf/progs/oom_policy.c -- 2.20.1