This is a note to let you know that I've just added the patch titled veth: Do not record rx queue hint in veth_xmit to the 5.15-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary The filename of the patch is: veth-do-not-record-rx-queue-hint-in-veth_xmit.patch and it can be found in the queue-5.15 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable@xxxxxxxxxxxxxxx> know about it. >From 710ad98c363a66a0cd8526465426c5c5f8377ee0 Mon Sep 17 00:00:00 2001 From: Daniel Borkmann <daniel@xxxxxxxxxxxxx> Date: Thu, 6 Jan 2022 01:46:06 +0100 Subject: veth: Do not record rx queue hint in veth_xmit From: Daniel Borkmann <daniel@xxxxxxxxxxxxx> commit 710ad98c363a66a0cd8526465426c5c5f8377ee0 upstream. Laurent reported that they have seen a significant amount of TCP retransmissions at high throughput from applications residing in network namespaces talking to the outside world via veths. The drops were seen on the qdisc layer (fq_codel, as per systemd default) of the phys device such as ena or virtio_net due to all traffic hitting a _single_ TX queue _despite_ multi-queue device. (Note that the setup was _not_ using XDP on veths as the issue is generic.) More specifically, after edbea9220251 ("veth: Store queue_mapping independently of XDP prog presence") which made it all the way back to v4.19.184+, skb_record_rx_queue() would set skb->queue_mapping to 1 (given 1 RX and 1 TX queue by default for veths) instead of leaving at 0. This is eventually retained and callbacks like ena_select_queue() will also pick single queue via netdev_core_pick_tx()'s ndo_select_queue() once all the traffic is forwarded to that device via upper stack or other means. Similarly, for others not implementing ndo_select_queue() if XPS is disabled, netdev_pick_tx() might call into the skb_tx_hash() and check for prior skb_rx_queue_recorded() as well. In general, it is a _bad_ idea for virtual devices like veth to mess around with queue selection [by default]. Given dev->real_num_tx_queues is by default 1, the skb->queue_mapping was left untouched, and so prior to edbea9220251 the netdev_core_pick_tx() could do its job upon __dev_queue_xmit() on the phys device. Unbreak this and restore prior behavior by removing the skb_record_rx_queue() from veth_xmit() altogether. If the veth peer has an XDP program attached, then it would return the first RX queue index in xdp_md->rx_queue_index (unless configured in non-default manner). However, this is still better than breaking the generic case. Fixes: edbea9220251 ("veth: Store queue_mapping independently of XDP prog presence") Fixes: 638264dc9022 ("veth: Support per queue XDP ring") Reported-by: Laurent Bernaille <laurent.bernaille@xxxxxxxxxxxxx> Signed-off-by: Daniel Borkmann <daniel@xxxxxxxxxxxxx> Cc: Maciej Fijalkowski <maciej.fijalkowski@xxxxxxxxx> Cc: Toshiaki Makita <toshiaki.makita1@xxxxxxxxx> Cc: Eric Dumazet <eric.dumazet@xxxxxxxxx> Cc: Paolo Abeni <pabeni@xxxxxxxxxx> Cc: John Fastabend <john.fastabend@xxxxxxxxx> Cc: Willem de Bruijn <willemb@xxxxxxxxxx> Acked-by: John Fastabend <john.fastabend@xxxxxxxxx> Reviewed-by: Eric Dumazet <edumazet@xxxxxxxxxx> Acked-by: Toshiaki Makita <toshiaki.makita1@xxxxxxxxx> Signed-off-by: David S. Miller <davem@xxxxxxxxxxxxx> Signed-off-by: Greg Kroah-Hartman <gregkh@xxxxxxxxxxxxxxxxxxx> --- drivers/net/veth.c | 1 - 1 file changed, 1 deletion(-) --- a/drivers/net/veth.c +++ b/drivers/net/veth.c @@ -342,7 +342,6 @@ static netdev_tx_t veth_xmit(struct sk_b */ use_napi = rcu_access_pointer(rq->napi) && veth_skb_is_eligible_for_gro(dev, rcv, skb); - skb_record_rx_queue(skb, rxq); } skb_tx_timestamp(skb); Patches currently in stable-queue which might be from daniel@xxxxxxxxxxxxx are queue-5.15/bpf-fix-out-of-bounds-access-from-invalid-_or_null-type-verification.patch queue-5.15/veth-do-not-record-rx-queue-hint-in-veth_xmit.patch