On Thu, Jan 21, 2021 at 11:54:24AM +0800, Hangbin Liu wrote:
> Hi Maciej,
> On Wed, Jan 20, 2021 at 11:42:38PM +0100, Maciej Fijalkowski wrote:
> > > +static int dev_map_bpf_prog_run(struct bpf_prog *xdp_prog,
> > > + struct xdp_frame **frames, int n,
> > > + struct net_device *dev)
> > > +{
> > > + struct xdp_txq_info txq = { .dev = dev };
> > > + struct xdp_buff xdp;
> > > + int i, nframes = 0;
> > > +
> > > + for (i = 0; i < n; i++) {
> > > + struct xdp_frame *xdpf = frames[i];
> > > + u32 act;
> > > + int err;
> > > +
> > > + xdp_convert_frame_to_buff(xdpf, &xdp);
> > > + xdp.txq = &txq;
> > > +
> > > + act = bpf_prog_run_xdp(xdp_prog, &xdp);
> > > + switch (act) {
> > > + case XDP_PASS:
> > > + err = xdp_update_frame_from_buff(&xdp, xdpf);
> >
> > Bump on John's question.
>
> Hi Jesper, would you please help answer John's question?
> > >
> > > - sent = dev->netdev_ops->ndo_xdp_xmit(dev, bq->count, bq->q, flags);
> > > + /* Init sent to cnt in case there is no xdp_prog */
> > > + sent = cnt;
> > > + if (bq->xdp_prog) {
> > > + sent = dev_map_bpf_prog_run(bq->xdp_prog, bq->q, cnt, dev);
> > > + if (!sent)
> > > + goto out;
> >
> > Sorry, but 'sent' is a bit confusing to me, actual sending happens below
> > via ndo_xdp_xmit, right? This hook will not actually send frames.
> > Can we do a subtle change to have it in separate variable 'to_send' ?
>
> Makes sense to me.
> >
> > Although I'm a huge goto advocate, I feel like this particular usage could
> > be simplified. Not sure why we had that in first place.
> >
> > I gave a shot at rewriting/refactoring whole bq_xmit_all and I feel like
> > it's more readable. I introduced 'to_send' variable and got rid of 'error'
> > label.
> >
> > Thoughts?
> >
> > I might have missed something, though.
> >
> > static void bq_xmit_all(struct xdp_dev_bulk_queue *bq, u32 flags)
> > {
> > struct net_device *dev = bq->dev;
> > unsigned int cnt = bq->count;
> > int drops = 0, err = 0;
> > int to_send = 0;
>
> The to_send also need to init to cnt.
So I missed something indeed :P you're correct
>
> > int sent = cnt;
> > int i;
> >
> > if (unlikely(!cnt))
> > return;
> >
> > for (i = 0; i < cnt; i++) {
> > struct xdp_frame *xdpf = bq->q[i];
> >
> > prefetch(xdpf);
> > }
> >
> > if (bq->xdp_prog) {
> > to_send = dev_map_bpf_prog_run(bq->xdp_prog, bq->q, cnt, dev);
> > if (!to_send) {
> > sent = 0;
> > goto out;
> > }
> > }
> >
> > drops = cnt - to_send;
>
> This line could move in to the xdp_prog brackets to save time when no xdp_prog.
Hmm, looks like we can do it.
For scenario where there was no bq->xdp_prog and failure of ndo_xdp_xmit,
we didn't alter the count of frames to be sent, so we would basically free
all of the frames (as drops is 0, cnt = bq->count). After that we
recalculate drops and correct value will be reported in tracepoint.
(needed to explain it to myself)
>
> if (bq->xdp_prog) {
> to_send = ...
> if (!to_send) {
> ...
> }
> drops = cnt - to_send;
> }
>
> > sent = dev->netdev_ops->ndo_xdp_xmit(dev, to_send, bq->q, flags);
>
> If we don't have xdp_prog, the to_send should be cnt.
Yes, we should init to_send to cnt as you're suggesting above.
>
> > if (sent < 0) {
> > err = sent;
> > sent = 0;
> >
> > /* If ndo_xdp_xmit fails with an errno, no frames have been
> > * xmit'ed and it's our responsibility to them free all.
> > */
> > for (i = 0; i < cnt - drops; i++) {
> > struct xdp_frame *xdpf = bq->q[i];
> >
> > xdp_return_frame_rx_napi(xdpf);
> > }
> > }
> > out:
> > drops = cnt - sent;
> > bq->count = 0;
> >
> > trace_xdp_devmap_xmit(bq->dev_rx, dev, sent, drops, err);
> > bq->dev_rx = NULL;
> > bq->xdp_prog = NULL;
> > __list_del_clearprev(&bq->flush_node);
> >
> > return;
> > }
>
> Thanks for your code, looks much clear now.
Good to hear! I agree on your points as well.
>
> Hangbin
