When NAPI mode is enabled, we try to free old transmited packets
before sending a packet. This has several side effects:
- transmitted packets need to be freed before sending a packet, this
introduces delay and increases the average packets transmit time.
- more time in hold the TX lock that causes more TX lock contention
with the TX NAPI
This would be more noticeable when using a fast device like
vhost-user/DPDK. So this patch tries to avoid those issues by not
cleaning transmitted packets in start_xmit() when TX NAPI is
enabled. Notification will be disabled at the beginning of the
start_xmit() but we can't enable delayed notification after TX is
stopped. Instead, the delayed notification needs to be enabled if we
need to kick the virtqueue.
Performance numbers:
1) pktgen_sample03_burst_single_flow.sh (burst 256) + testpmd (rxonly)
on the host:
- When pinning TX IRQ to pktgen VCPU: split virtqueue PPS were
increased 62% from 6.45 Mpps to 10.5 Mpps; packed virtqueue PPS were
increased 60% from 7.8 Mpps to 12.5 Mpps.
- When pinning TX IRQ to VCPU other than pktgen: split virtqueue PPS
were increased 25% from 6.15 Mpps to 7.7 Mpps; packed virtqueue PPS
were increased 50.6% from 8.3Mpps to 12.5 Mpps.
2) Netperf:
- Netperf in guest + vhost-net/TAP on the host doesn't show obvious
differences.
Signed-off-by: Jason Wang <jasowang@xxxxxxxxxx>
---
drivers/net/virtio_net.c | 27 ++++++++++++++++++---------
1 file changed, 18 insertions(+), 9 deletions(-)
diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index 2f6c3dc68ba0..3d5c44546dc1 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -3271,15 +3271,10 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
bool use_napi = sq->napi.weight;
bool kick;
- /* Free up any pending old buffers before queueing new ones. */
- do {
- if (use_napi)
- virtqueue_disable_cb(sq->vq);
-
+ if (!use_napi)
free_old_xmit(sq, txq, false);
-
- } while (use_napi && !xmit_more &&
- unlikely(!virtqueue_enable_cb_delayed(sq->vq)));
+ else
+ virtqueue_disable_cb(sq->vq);
/* timestamp packet in software */
skb_tx_timestamp(skb);
@@ -3305,7 +3300,18 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
nf_reset_ct(skb);
}
- check_sq_full_and_disable(vi, dev, sq);
+ if (tx_may_stop(vi, dev, sq) && !use_napi &&
+ unlikely(virtqueue_enable_cb_delayed(sq->vq))) {
+ /* More just got used, free them then recheck. */
+ free_old_xmit(sq, txq, false);
+ if (sq->vq->num_free >= 2+MAX_SKB_FRAGS) {
+ netif_start_subqueue(dev, qnum);
+ u64_stats_update_begin(&sq->stats.syncp);
+ u64_stats_inc(&sq->stats.wake);
+ u64_stats_update_end(&sq->stats.syncp);
+ virtqueue_disable_cb(sq->vq);
+ }
+ }
kick = use_napi ? __netdev_tx_sent_queue(txq, skb->len, xmit_more) :
!xmit_more || netif_xmit_stopped(txq);
@@ -3317,6 +3323,9 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
}
}
+ if (use_napi && kick && unlikely(!virtqueue_enable_cb_delayed(sq->vq)))
+ virtqueue_napi_schedule(&sq->napi, sq->vq);
+
return NETDEV_TX_OK;
}
--
2.34.1
