On Mon, Mar 22, 2021 at 10:00:33PM +0200, Vladimir Oltean wrote: > Hi Yunsheng, > > On Mon, Mar 22, 2021 at 05:09:16PM +0800, Yunsheng Lin wrote: > > Currently pfifo_fast has both TCQ_F_CAN_BYPASS and TCQ_F_NOLOCK > > flag set, but queue discipline by-pass does not work for lockless > > qdisc because skb is always enqueued to qdisc even when the qdisc > > is empty, see __dev_xmit_skb(). > > > > This patch calls sch_direct_xmit() to transmit the skb directly > > to the driver for empty lockless qdisc too, which aviod enqueuing > > and dequeuing operation. qdisc->empty is set to false whenever a > > skb is enqueued, see pfifo_fast_enqueue(), and is set to true when > > skb dequeuing return NULL, see pfifo_fast_dequeue(). > > > > There is a data race between enqueue/dequeue and qdisc->empty > > setting, qdisc->empty is only used as a hint, so we need to call > > sch_may_need_requeuing() to see if the queue is really empty and if > > there is requeued skb, which has higher priority than the current > > skb. > > > > The performance for ip_forward test increases about 10% with this > > patch. > > > > Signed-off-by: Yunsheng Lin <linyunsheng@xxxxxxxxxx> > > --- > > Hi, Vladimir and Ahmad > > Please give it a test to see if there is any out of order > > packet for this patch, which has removed the priv->lock added in > > RFC v2. > > > > There is a data race as below: > > > > CPU1 CPU2 > > qdisc_run_begin(q) . > > . q->enqueue() > > sch_may_need_requeuing() . > > return true . > > . . > > . . > > q->enqueue() . > > > > When above happen, the skb enqueued by CPU1 is dequeued after the > > skb enqueued by CPU2 because sch_may_need_requeuing() return true. > > If there is not qdisc bypass, the CPU1 has better chance to queue > > the skb quicker than CPU2. > > > > This patch does not take care of the above data race, because I > > view this as similar as below: > > > > Even at the same time CPU1 and CPU2 write the skb to two socket > > which both heading to the same qdisc, there is no guarantee that > > which skb will hit the qdisc first, becuase there is a lot of > > factor like interrupt/softirq/cache miss/scheduling afffecting > > that. > > > > So I hope the above data race will not cause problem for Vladimir > > and Ahmad. > > --- > > Preliminary results on my test setup look fine, but please allow me to > run the canfdtest overnight, since as you say, races are still > theoretically possible. I haven't found any issues during the overnight test and until now. Tested-by: Vladimir Oltean <vladimir.oltean@xxxxxxx> # flexcan
