xdp_do_flush() should be invoked before leaving the NAPI poll function
if XDP-redirect has been performed.
There are two possible XDP invocations in cpu_map_bpf_prog_run():
- cpu_map_bpf_prog_run_xdp()
- cpu_map_bpf_prog_run_skb()
Both functions update stats->redirect if a redirect is performed but
xdp_do_flush() is only invoked after the former.
Invoke xdp_do_flush() after both functions run and a redirect was
performed.
Cc: Andrii Nakryiko <andrii@xxxxxxxxxx>
Cc: Hao Luo <haoluo@xxxxxxxxxx>
Cc: Jiri Olsa <jolsa@xxxxxxxxxx>
Cc: KP Singh <kpsingh@xxxxxxxxxx>
Cc: Martin KaFai Lau <martin.lau@xxxxxxxxx>
Cc: Song Liu <song@xxxxxxxxxx>
Cc: Stanislav Fomichev <sdf@xxxxxxxxxx>
Cc: Yonghong Song <yonghong.song@xxxxxxxxx>
Fixes: 11941f8a85362 ("bpf: cpumap: Implement generic cpumap")
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@xxxxxxxxxxxxx>
---
kernel/bpf/cpumap.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/kernel/bpf/cpumap.c b/kernel/bpf/cpumap.c
index e42a1bdb7f536..f3ba11b4a732b 100644
--- a/kernel/bpf/cpumap.c
+++ b/kernel/bpf/cpumap.c
@@ -248,12 +248,12 @@ static int cpu_map_bpf_prog_run(struct bpf_cpu_map_entry *rcpu, void **frames,
nframes = cpu_map_bpf_prog_run_xdp(rcpu, frames, xdp_n, stats);
- if (stats->redirect)
- xdp_do_flush();
-
if (unlikely(!list_empty(list)))
cpu_map_bpf_prog_run_skb(rcpu, list, stats);
+ if (stats->redirect)
+ xdp_do_flush();
+
rcu_read_unlock_bh(); /* resched point, may call do_softirq() */
return nframes;
--
2.40.1
