Patch "bpf_lru_list: Read double-checked variable once without lock" has been added to the 5.4-stable tree

[Sasha Levin <sashal@xxxxxxxxxx>]

This is a note to let you know that I've just added the patch titled

    bpf_lru_list: Read double-checked variable once without lock

to the 5.4-stable tree which can be found at:
    http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
     bpf_lru_list-read-double-checked-variable-once-witho.patch
and it can be found in the queue-5.4 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree,
please let <stable@xxxxxxxxxxxxxxx> know about it.



commit 4458d878659efb4360f88e7bcd8ca5bd7bc1a0b9
Author: Marco Elver <elver@xxxxxxxxxx>
Date:   Tue Feb 9 12:27:01 2021 +0100

    bpf_lru_list: Read double-checked variable once without lock
    
    [ Upstream commit 6df8fb83301d68ea0a0c0e1cbcc790fcc333ed12 ]
    
    For double-checked locking in bpf_common_lru_push_free(), node->type is
    read outside the critical section and then re-checked under the lock.
    However, concurrent writes to node->type result in data races.
    
    For example, the following concurrent access was observed by KCSAN:
    
      write to 0xffff88801521bc22 of 1 bytes by task 10038 on cpu 1:
       __bpf_lru_node_move_in        kernel/bpf/bpf_lru_list.c:91
       __local_list_flush            kernel/bpf/bpf_lru_list.c:298
       ...
      read to 0xffff88801521bc22 of 1 bytes by task 10043 on cpu 0:
       bpf_common_lru_push_free      kernel/bpf/bpf_lru_list.c:507
       bpf_lru_push_free             kernel/bpf/bpf_lru_list.c:555
       ...
    
    Fix the data races where node->type is read outside the critical section
    (for double-checked locking) by marking the access with READ_ONCE() as
    well as ensuring the variable is only accessed once.
    
    Fixes: 3a08c2fd7634 ("bpf: LRU List")
    Reported-by: syzbot+3536db46dfa58c573458@xxxxxxxxxxxxxxxxxxxxxxxxx
    Reported-by: syzbot+516acdb03d3e27d91bcd@xxxxxxxxxxxxxxxxxxxxxxxxx
    Signed-off-by: Marco Elver <elver@xxxxxxxxxx>
    Signed-off-by: Andrii Nakryiko <andrii@xxxxxxxxxx>
    Acked-by: Martin KaFai Lau <kafai@xxxxxx>
    Link: https://lore.kernel.org/bpf/20210209112701.3341724-1-elver@xxxxxxxxxx
    Signed-off-by: Sasha Levin <sashal@xxxxxxxxxx>

diff --git a/kernel/bpf/bpf_lru_list.c b/kernel/bpf/bpf_lru_list.c
index 1b6b9349cb857..d99e89f113c43 100644
--- a/kernel/bpf/bpf_lru_list.c
+++ b/kernel/bpf/bpf_lru_list.c
@@ -502,13 +502,14 @@ struct bpf_lru_node *bpf_lru_pop_free(struct bpf_lru *lru, u32 hash)
 static void bpf_common_lru_push_free(struct bpf_lru *lru,
 				     struct bpf_lru_node *node)
 {
+	u8 node_type = READ_ONCE(node->type);
 	unsigned long flags;
 
-	if (WARN_ON_ONCE(node->type == BPF_LRU_LIST_T_FREE) ||
-	    WARN_ON_ONCE(node->type == BPF_LRU_LOCAL_LIST_T_FREE))
+	if (WARN_ON_ONCE(node_type == BPF_LRU_LIST_T_FREE) ||
+	    WARN_ON_ONCE(node_type == BPF_LRU_LOCAL_LIST_T_FREE))
 		return;
 
-	if (node->type == BPF_LRU_LOCAL_LIST_T_PENDING) {
+	if (node_type == BPF_LRU_LOCAL_LIST_T_PENDING) {
 		struct bpf_lru_locallist *loc_l;
 
 		loc_l = per_cpu_ptr(lru->common_lru.local_list, node->cpu);