Hi,
Just to confirm the two patches provided also fixed our internal software as well.
I’ve seen the patches have made their way to the mainline code so thank you again.
Would you like us to test the 4k ring size fix still?
KR
Alasdair
> On 12 Aug 2022, at 11:37, Maciej Fijalkowski <maciej.fijalkowski@xxxxxxxxx> wrote:
>
> On Fri, Aug 12, 2022 at 10:29:28AM +0000, Alasdair McWilliam wrote:
>> Maciej, Magnus,
>>
>>>> I have just sent patches and CCed you on them which should address the
>>>> issue you are observing. I will be grateful if you could test them on your
>>>> side and get back to us with results. If this won't help then we'll need
>>>> to dig this more.
>>
>>>> What ring size you're using?
>>
>> I’ve applied both of those patches on top of the patch from Magnus the
>> other day (including the if (!pool->unaligned) tweak below).
>>
>> With both NIC tx and rx rings at 4096, using xdpsock_multi, all queues
>> now seem to see process packets, which is good, but only up to ~4096
>> packets before stalling.
>>
>> However, your additional question about ring size got me to try a tx/rx
>> ring size of 2048 each, and it springs to life! 2Mpps l2fwd per queue on
>> 8 queues with xdpsock_multi. Repeated with 4 queues, 2 queues etc and
>> all looking good.
>
> Awesome!
>
>>
>> Seems ICE doesn’t like a 4096 ring size atm. :-)
>
> I have a fix for that, but please give me few days to clean this up. I
> will be able to share it with you next week and I would really appreciate
> if you could test this as well.
>
>>
>>
>>> Thank you so much for testing Alasdair. I believe your tests are
>>> successful in terms of not getting corrupted packets anymore, which is
>>> the only thing the patch I sent will fix. So we still have two
>>> problems to fix for you:
>>>
>>> 1: RSS not working correctly for E810, which is what Maciej is
>>> addressing in his patch set
>>> 2: The queue stalls after 4K packets.
>>>
>>> I can take a look at #2 using the application you sent, but after
>>> applying my patch and Maciej's.
>>
>>
>>>>>> static int xp_init_dma_info(struct xsk_buff_pool *pool, struct xsk_dma_map *dma_map)
>>>>>> {
>>>>>> + u32 i;
>>>>>> +
>>>>>> + for (i = 0; i < pool->heads_cnt; i++) {
>>>>>> + struct xdp_buff_xsk *xskb = &pool->heads[i];
>>>>>> +
>>>>>> + xp_init_xskb_dma(xskb, pool, dma_map->dma_pages, xskb->orig_addr);
>>>>>> + }
>>>>>> +
>>>
>>> This for loop needs to be protected with an if (!pool->unaligned), but
>>> I will not send out a new version here. It will be in the version sent
>>> to the netdev mailing list.
>>
>> To confirm I applied this as well before applying Maciej’s patches.
>>
>> Now we can now run xdpsock_multi with ring sizes at 2048 okay, I’ll spin
>> up our software stack on this patched kernel to validate that side of
>> things too.
>
> Glad to hear that! I'll ping you once I have a fix for 4k rings.
>
> Thanks,
> Maciej
>
>>
>> Thank you both!!
>> Alasdair
|