On Mon, 11 Nov 2024, Liu Jian wrote:
> BUG: KASAN: slab-use-after-free in tcp_write_timer_handler+0x156/0x3e0
> Read of size 1 at addr ffff888111f322cd by task swapper/0/0
>
> CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Not tainted 6.12.0-rc4-dirty #7
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1
> Call Trace:
> <IRQ>
> dump_stack_lvl+0x68/0xa0
> print_address_description.constprop.0+0x2c/0x3d0
> print_report+0xb4/0x270
> kasan_report+0xbd/0xf0
> tcp_write_timer_handler+0x156/0x3e0
> tcp_write_timer+0x66/0x170
> call_timer_fn+0xfb/0x1d0
> __run_timers+0x3f8/0x480
> run_timer_softirq+0x9b/0x100
> handle_softirqs+0x153/0x390
> __irq_exit_rcu+0x103/0x120
> irq_exit_rcu+0xe/0x20
> sysvec_apic_timer_interrupt+0x76/0x90
> </IRQ>
> <TASK>
> asm_sysvec_apic_timer_interrupt+0x1a/0x20
> RIP: 0010:default_idle+0xf/0x20
> Code: 4c 01 c7 4c 29 c2 e9 72 ff ff ff 90 90 90 90 90 90 90 90 90 90 90 90
> 90 90 90 90 f3 0f 1e fa 66 90 0f 00 2d 33 f8 25 00 fb f4 <fa> c3 cc cc cc
> cc 66 66 2e 0f 1f 84 00 00 00 00 00 90 90 90 90 90
> RSP: 0018:ffffffffa2007e28 EFLAGS: 00000242
> RAX: 00000000000f3b31 RBX: 1ffffffff4400fc7 RCX: ffffffffa09c3196
> RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff9f00590f
> RBP: 0000000000000000 R08: 0000000000000001 R09: ffffed102360835d
> R10: ffff88811b041aeb R11: 0000000000000001 R12: 0000000000000000
> R13: ffffffffa202d7c0 R14: 0000000000000000 R15: 00000000000147d0
> default_idle_call+0x6b/0xa0
> cpuidle_idle_call+0x1af/0x1f0
> do_idle+0xbc/0x130
> cpu_startup_entry+0x33/0x40
> rest_init+0x11f/0x210
> start_kernel+0x39a/0x420
> x86_64_start_reservations+0x18/0x30
> x86_64_start_kernel+0x97/0xa0
> common_startup_64+0x13e/0x141
> </TASK>
>
> Allocated by task 595:
> kasan_save_stack+0x24/0x50
> kasan_save_track+0x14/0x30
> __kasan_slab_alloc+0x87/0x90
> kmem_cache_alloc_noprof+0x12b/0x3f0
> copy_net_ns+0x94/0x380
> create_new_namespaces+0x24c/0x500
> unshare_nsproxy_namespaces+0x75/0xf0
> ksys_unshare+0x24e/0x4f0
> __x64_sys_unshare+0x1f/0x30
> do_syscall_64+0x70/0x180
> entry_SYSCALL_64_after_hwframe+0x76/0x7e
>
> Freed by task 100:
> kasan_save_stack+0x24/0x50
> kasan_save_track+0x14/0x30
> kasan_save_free_info+0x3b/0x60
> __kasan_slab_free+0x54/0x70
> kmem_cache_free+0x156/0x5d0
> cleanup_net+0x5d3/0x670
> process_one_work+0x776/0xa90
> worker_thread+0x2e2/0x560
> kthread+0x1a8/0x1f0
> ret_from_fork+0x34/0x60
> ret_from_fork_asm+0x1a/0x30
>
> Reproduction script:
>
> mkdir -p /mnt/nfsshare
> mkdir -p /mnt/nfs/netns_1
> mkfs.ext4 /dev/sdb
> mount /dev/sdb /mnt/nfsshare
> systemctl restart nfs-server
> chmod 777 /mnt/nfsshare
> exportfs -i -o rw,no_root_squash *:/mnt/nfsshare
>
> ip netns add netns_1
> ip link add name veth_1_peer type veth peer veth_1
> ifconfig veth_1_peer 11.11.0.254 up
> ip link set veth_1 netns netns_1
> ip netns exec netns_1 ifconfig veth_1 11.11.0.1
>
> ip netns exec netns_1 /root/iptables -A OUTPUT -d 11.11.0.254 -p tcp \
> --tcp-flags FIN FIN -j DROP
>
> (note: In my environment, a DESTROY_CLIENTID operation is always sent
> immediately, breaking the nfs tcp connection.)
> ip netns exec netns_1 timeout -s 9 300 mount -t nfs -o proto=tcp,vers=4.1 \
> 11.11.0.254:/mnt/nfsshare /mnt/nfs/netns_1
>
> ip netns del netns_1
>
> The reason here is that the tcp socket in netns_1 (nfs side) has been
> shutdown and closed (done in xs_destroy), but the FIN message (with ack)
> is discarded, and the nfsd side keeps sending retransmission messages.
> As a result, when the tcp sock in netns_1 processes the received message,
> it sends the message (FIN message) in the sending queue, and the tcp timer
> is re-established. When the network namespace is deleted, the net structure
> accessed by tcp's timer handler function causes problems.
>
> To fix this problem, let's hold netns refcnt for the tcp kernel socket as
> done in other modules.
>
> Fixes: 26abe14379f8 ("net: Modify sk_alloc to not reference count the netns of kernel sockets.")
> Signed-off-by: Liu Jian <liujian56@xxxxxxxxxx>
> ---
> net/sunrpc/svcsock.c | 4 ++++
> net/sunrpc/xprtsock.c | 6 ++++++
> 2 files changed, 10 insertions(+)
>
> diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c
> index 6f272013fd9b..d4330aaadc23 100644
> --- a/net/sunrpc/svcsock.c
> +++ b/net/sunrpc/svcsock.c
> @@ -1551,6 +1551,10 @@ static struct svc_xprt *svc_create_socket(struct svc_serv *serv,
> newlen = error;
>
> if (protocol == IPPROTO_TCP) {
> + __netns_tracker_free(net, &sock->sk->ns_tracker, false);
> + sock->sk->sk_net_refcnt = 1;
> + get_net_track(net, &sock->sk->ns_tracker, GFP_KERNEL);
> + sock_inuse_add(net, 1);
This is really ugly. These internal details of the network layer have
no place in sunrpc code. There must be a better way.
Can we pass '0' for the kern arg to __sock_create()? That should fix
the refcounting issues, but might mess up security labelling.
Can we wait for something before we call put_net() to release the net.
Maybe we want to split the "kern" arg t __sock_create() and have
"kern" which affects labeling and "refnet" with affects refcounting the
net.
I had a quick look and very nearly every caller of __sock_create()
outside of net/core really does want refcount. Many callers of
sock_create_kern() possibly don't.
So I really think this needs to be cleaned up in net/core, not in all
the different network clients in the kernel.
Thanks,
NeilBrown
> if ((error = kernel_listen(sock, 64)) < 0)
> goto bummer;
> }
> diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c
> index d2f31b59457b..0f0b9f9283d9 100644
> --- a/net/sunrpc/xprtsock.c
> +++ b/net/sunrpc/xprtsock.c
> @@ -1942,6 +1942,12 @@ static struct socket *xs_create_sock(struct rpc_xprt *xprt,
> goto out;
> }
>
> + if (protocol == IPPROTO_TCP) {
> + __netns_tracker_free(xprt->xprt_net, &sock->sk->ns_tracker, false);
> + sock->sk->sk_net_refcnt = 1;
> + get_net_track(xprt->xprt_net, &sock->sk->ns_tracker, GFP_KERNEL);
> + sock_inuse_add(xprt->xprt_net, 1);
> + }
> filp = sock_alloc_file(sock, O_NONBLOCK, NULL);
> if (IS_ERR(filp))
> return ERR_CAST(filp);
> --
> 2.34.1
>
>
