Maciej, Magnus,
>> I have just sent patches and CCed you on them which should address the
>> issue you are observing. I will be grateful if you could test them on your
>> side and get back to us with results. If this won't help then we'll need
>> to dig this more.
>> What ring size you're using?
I’ve applied both of those patches on top of the patch from Magnus the other day (including the if (!pool->unaligned) tweak below).
With both NIC tx and rx rings at 4096, using xdpsock_multi, all queues now seem to see process packets, which is good, but only up to ~4096 packets before stalling.
However, your additional question about ring size got me to try a tx/rx ring size of 2048 each, and it springs to life! 2Mpps l2fwd per queue on 8 queues with xdpsock_multi. Repeated with 4 queues, 2 queues etc and all looking good.
Seems ICE doesn’t like a 4096 ring size atm. :-)
> Thank you so much for testing Alasdair. I believe your tests are
> successful in terms of not getting corrupted packets anymore, which is
> the only thing the patch I sent will fix. So we still have two
> problems to fix for you:
>
> 1: RSS not working correctly for E810, which is what Maciej is
> addressing in his patch set
> 2: The queue stalls after 4K packets.
>
> I can take a look at #2 using the application you sent, but after
> applying my patch and Maciej's.
>>>> static int xp_init_dma_info(struct xsk_buff_pool *pool, struct xsk_dma_map *dma_map)
>>>> {
>>>> + u32 i;
>>>> +
>>>> + for (i = 0; i < pool->heads_cnt; i++) {
>>>> + struct xdp_buff_xsk *xskb = &pool->heads[i];
>>>> +
>>>> + xp_init_xskb_dma(xskb, pool, dma_map->dma_pages, xskb->orig_addr);
>>>> + }
>>>> +
>
> This for loop needs to be protected with an if (!pool->unaligned), but
> I will not send out a new version here. It will be in the version sent
> to the netdev mailing list.
To confirm I applied this as well before applying Maciej’s patches.
Now we can now run xdpsock_multi with ring sizes at 2048 okay, I’ll spin up our software stack on this patched kernel to validate that side of things too.
Thank you both!!
Alasdair
|