Patch "bpf: Use this_cpu_{inc|dec|inc_return} for bpf_task_storage_busy" has been added to the 5.15-stable tree

[Sasha Levin <sashal@xxxxxxxxxx>]

This is a note to let you know that I've just added the patch titled

    bpf: Use this_cpu_{inc|dec|inc_return} for bpf_task_storage_busy

to the 5.15-stable tree which can be found at:
    http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
     bpf-use-this_cpu_-inc-dec-inc_return-for-bpf_task_st.patch
and it can be found in the queue-5.15 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree,
please let <stable@xxxxxxxxxxxxxxx> know about it.



commit 9cc57db8711eb7d4e369a04e19491706312f7cc8
Author: Hou Tao <houtao1@xxxxxxxxxx>
Date:   Thu Sep 1 14:19:35 2022 +0800

    bpf: Use this_cpu_{inc|dec|inc_return} for bpf_task_storage_busy
    
    [ Upstream commit 197827a05e13808c60f52632e9887eede63f1c16 ]
    
    Now migrate_disable() does not disable preemption and under some
    architectures (e.g. arm64) __this_cpu_{inc|dec|inc_return} are neither
    preemption-safe nor IRQ-safe, so for fully preemptible kernel concurrent
    lookups or updates on the same task local storage and on the same CPU
    may make bpf_task_storage_busy be imbalanced, and
    bpf_task_storage_trylock() on the specific cpu will always fail.
    
    Fixing it by using this_cpu_{inc|dec|inc_return} when manipulating
    bpf_task_storage_busy.
    
    Fixes: bc235cdb423a ("bpf: Prevent deadlock from recursive bpf_task_storage_[get|delete]")
    Signed-off-by: Hou Tao <houtao1@xxxxxxxxxx>
    Acked-by: Alexei Starovoitov <ast@xxxxxxxxxx>
    Link: https://lore.kernel.org/r/20220901061938.3789460-2-houtao@xxxxxxxxxxxxxxx
    Signed-off-by: Martin KaFai Lau <martin.lau@xxxxxxxxxx>
    Signed-off-by: Sasha Levin <sashal@xxxxxxxxxx>

diff --git a/kernel/bpf/bpf_local_storage.c b/kernel/bpf/bpf_local_storage.c
index b305270b7a4b..de4d741d99a3 100644
--- a/kernel/bpf/bpf_local_storage.c
+++ b/kernel/bpf/bpf_local_storage.c
@@ -506,11 +506,11 @@ void bpf_local_storage_map_free(struct bpf_local_storage_map *smap,
 				struct bpf_local_storage_elem, map_node))) {
 			if (busy_counter) {
 				migrate_disable();
-				__this_cpu_inc(*busy_counter);
+				this_cpu_inc(*busy_counter);
 			}
 			bpf_selem_unlink(selem);
 			if (busy_counter) {
-				__this_cpu_dec(*busy_counter);
+				this_cpu_dec(*busy_counter);
 				migrate_enable();
 			}
 			cond_resched_rcu();
diff --git a/kernel/bpf/bpf_task_storage.c b/kernel/bpf/bpf_task_storage.c
index ebfa8bc90892..6b7bfce23915 100644
--- a/kernel/bpf/bpf_task_storage.c
+++ b/kernel/bpf/bpf_task_storage.c
@@ -25,20 +25,20 @@ static DEFINE_PER_CPU(int, bpf_task_storage_busy);
 static void bpf_task_storage_lock(void)
 {
 	migrate_disable();
-	__this_cpu_inc(bpf_task_storage_busy);
+	this_cpu_inc(bpf_task_storage_busy);
 }
 
 static void bpf_task_storage_unlock(void)
 {
-	__this_cpu_dec(bpf_task_storage_busy);
+	this_cpu_dec(bpf_task_storage_busy);
 	migrate_enable();
 }
 
 static bool bpf_task_storage_trylock(void)
 {
 	migrate_disable();
-	if (unlikely(__this_cpu_inc_return(bpf_task_storage_busy) != 1)) {
-		__this_cpu_dec(bpf_task_storage_busy);
+	if (unlikely(this_cpu_inc_return(bpf_task_storage_busy) != 1)) {
+		this_cpu_dec(bpf_task_storage_busy);
 		migrate_enable();
 		return false;
 	}