Patch "bpf: Zeroing allocated object from slab in bpf memory allocator" has been added to the 6.2-stable tree

[Sasha Levin <sashal@xxxxxxxxxx>]

This is a note to let you know that I've just added the patch titled

    bpf: Zeroing allocated object from slab in bpf memory allocator

to the 6.2-stable tree which can be found at:
    http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
     bpf-zeroing-allocated-object-from-slab-in-bpf-memory.patch
and it can be found in the queue-6.2 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree,
please let <stable@xxxxxxxxxxxxxxx> know about it.



commit 23fa5f7877337506fad1db8daeaa3e4fe9b1d0fd
Author: Hou Tao <houtao1@xxxxxxxxxx>
Date:   Wed Feb 15 16:21:31 2023 +0800

    bpf: Zeroing allocated object from slab in bpf memory allocator
    
    [ Upstream commit 997849c4b969034e225153f41026657def66d286 ]
    
    Currently the freed element in bpf memory allocator may be immediately
    reused, for htab map the reuse will reinitialize special fields in map
    value (e.g., bpf_spin_lock), but lookup procedure may still access
    these special fields, and it may lead to hard-lockup as shown below:
    
     NMI backtrace for cpu 16
     CPU: 16 PID: 2574 Comm: htab.bin Tainted: G             L     6.1.0+ #1
     Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
     RIP: 0010:queued_spin_lock_slowpath+0x283/0x2c0
     ......
     Call Trace:
      <TASK>
      copy_map_value_locked+0xb7/0x170
      bpf_map_copy_value+0x113/0x3c0
      __sys_bpf+0x1c67/0x2780
      __x64_sys_bpf+0x1c/0x20
      do_syscall_64+0x30/0x60
      entry_SYSCALL_64_after_hwframe+0x46/0xb0
     ......
      </TASK>
    
    For htab map, just like the preallocated case, these is no need to
    initialize these special fields in map value again once these fields
    have been initialized. For preallocated htab map, these fields are
    initialized through __GFP_ZERO in bpf_map_area_alloc(), so do the
    similar thing for non-preallocated htab in bpf memory allocator. And
    there is no need to use __GFP_ZERO for per-cpu bpf memory allocator,
    because __alloc_percpu_gfp() does it implicitly.
    
    Fixes: 0fd7c5d43339 ("bpf: Optimize call_rcu in non-preallocated hash map.")
    Signed-off-by: Hou Tao <houtao1@xxxxxxxxxx>
    Link: https://lore.kernel.org/r/20230215082132.3856544-2-houtao@xxxxxxxxxxxxxxx
    Signed-off-by: Alexei Starovoitov <ast@xxxxxxxxxx>
    Signed-off-by: Sasha Levin <sashal@xxxxxxxxxx>

diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 634d37a599fa7..cf0d88109e3f9 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -346,6 +346,13 @@ static inline void bpf_obj_init(const struct btf_field_offs *foffs, void *obj)
 		memset(obj + foffs->field_off[i], 0, foffs->field_sz[i]);
 }
 
+/* 'dst' must be a temporary buffer and should not point to memory that is being
+ * used in parallel by a bpf program or bpf syscall, otherwise the access from
+ * the bpf program or bpf syscall may be corrupted by the reinitialization,
+ * leading to weird problems. Even 'dst' is newly-allocated from bpf memory
+ * allocator, it is still possible for 'dst' to be used in parallel by a bpf
+ * program or bpf syscall.
+ */
 static inline void check_and_init_map_value(struct bpf_map *map, void *dst)
 {
 	bpf_obj_init(map->field_offs, dst);
diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c
index 66bded1443773..5dfcb5ad0d068 100644
--- a/kernel/bpf/hashtab.c
+++ b/kernel/bpf/hashtab.c
@@ -1004,8 +1004,6 @@ static struct htab_elem *alloc_htab_elem(struct bpf_htab *htab, void *key,
 			l_new = ERR_PTR(-ENOMEM);
 			goto dec_count;
 		}
-		check_and_init_map_value(&htab->map,
-					 l_new->key + round_up(key_size, 8));
 	}
 
 	memcpy(l_new->key, key, key_size);
@@ -1592,6 +1590,7 @@ static int __htab_map_lookup_and_delete_elem(struct bpf_map *map, void *key,
 			else
 				copy_map_value(map, value, l->key +
 					       roundup_key_size);
+			/* Zeroing special fields in the temp buffer */
 			check_and_init_map_value(map, value);
 		}
 
@@ -1792,6 +1791,7 @@ __htab_map_lookup_and_delete_batch(struct bpf_map *map,
 						      true);
 			else
 				copy_map_value(map, dst_val, value);
+			/* Zeroing special fields in the temp buffer */
 			check_and_init_map_value(map, dst_val);
 		}
 		if (do_delete) {
diff --git a/kernel/bpf/memalloc.c b/kernel/bpf/memalloc.c
index 1db156405b68b..7b784823a52ef 100644
--- a/kernel/bpf/memalloc.c
+++ b/kernel/bpf/memalloc.c
@@ -143,7 +143,7 @@ static void *__alloc(struct bpf_mem_cache *c, int node)
 		return obj;
 	}
 
-	return kmalloc_node(c->unit_size, flags, node);
+	return kmalloc_node(c->unit_size, flags | __GFP_ZERO, node);
 }
 
 static struct mem_cgroup *get_memcg(const struct bpf_mem_cache *c)