On Fri, Aug 12, 2022 at 10:29:28AM +0000, Alasdair McWilliam wrote:
> Maciej, Magnus,
>
> >> I have just sent patches and CCed you on them which should address the
> >> issue you are observing. I will be grateful if you could test them on your
> >> side and get back to us with results. If this won't help then we'll need
> >> to dig this more.
>
> >> What ring size you're using?
>
> I’ve applied both of those patches on top of the patch from Magnus the
> other day (including the if (!pool->unaligned) tweak below).
>
> With both NIC tx and rx rings at 4096, using xdpsock_multi, all queues
> now seem to see process packets, which is good, but only up to ~4096
> packets before stalling.
>
> However, your additional question about ring size got me to try a tx/rx
> ring size of 2048 each, and it springs to life! 2Mpps l2fwd per queue on
> 8 queues with xdpsock_multi. Repeated with 4 queues, 2 queues etc and
> all looking good.
Awesome!
>
> Seems ICE doesn’t like a 4096 ring size atm. :-)
I have a fix for that, but please give me few days to clean this up. I
will be able to share it with you next week and I would really appreciate
if you could test this as well.
>
>
> > Thank you so much for testing Alasdair. I believe your tests are
> > successful in terms of not getting corrupted packets anymore, which is
> > the only thing the patch I sent will fix. So we still have two
> > problems to fix for you:
> >
> > 1: RSS not working correctly for E810, which is what Maciej is
> > addressing in his patch set
> > 2: The queue stalls after 4K packets.
> >
> > I can take a look at #2 using the application you sent, but after
> > applying my patch and Maciej's.
>
>
> >>>> static int xp_init_dma_info(struct xsk_buff_pool *pool, struct xsk_dma_map *dma_map)
> >>>> {
> >>>> + u32 i;
> >>>> +
> >>>> + for (i = 0; i < pool->heads_cnt; i++) {
> >>>> + struct xdp_buff_xsk *xskb = &pool->heads[i];
> >>>> +
> >>>> + xp_init_xskb_dma(xskb, pool, dma_map->dma_pages, xskb->orig_addr);
> >>>> + }
> >>>> +
> >
> > This for loop needs to be protected with an if (!pool->unaligned), but
> > I will not send out a new version here. It will be in the version sent
> > to the netdev mailing list.
>
> To confirm I applied this as well before applying Maciej’s patches.
>
> Now we can now run xdpsock_multi with ring sizes at 2048 okay, I’ll spin
> up our software stack on this patched kernel to validate that side of
> things too.
Glad to hear that! I'll ping you once I have a fix for 4k rings.
Thanks,
Maciej
>
> Thank you both!!
> Alasdair
>
|