Re: Is It Possible to RX/Process/TX packets concurrently with AF_XDP?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



It looks like that I didn’t include the mailing list in my previous replies. I hope this one does.

Also, for the AF_XDP-forwarding example, is it able to handle multiple AF_XDP sockets on the same NIC?

Such as:

```
./xdp_fwd -i IFA -q Q1 -i IFA -q Q2 -i IFA -q Q3 -i IFA -q Q4 -c CX -c CY

```

If the above is doable, maybe I can have multiple queues, rather than having one, on the same NIC, create one AF_XDP socket per queue, and then use this xdp_fwd example to achieve multi-threading?

Thank you very much for your help and time.
Rio 


From: Zhaoxi Zhu <zzhu@xxxxxxxxxxxxx>
Date: Monday, December 12, 2022 at 11:06 AM
To: Magnus Karlsson <magnus.karlsson@xxxxxxxxx>
Subject: Re: Is It Possible to RX/Process/TX packets concurrently with AF_XDP?

Got it, thank you very much for your clarification.

I have one more question, if I may: If one AF_XDP should be handled by one thread, in order to avoid mutexes and to achieve better performance, then, can I have more than one AF_XDP socket on the same physical NIC, and use one thread per AF_XDP socket, in order to make process packets coming into this NIC concurrently?

Currently, the way we are testing AF_XDP with is to have only 1 queue:

```
sudo ethtool -L <interface> combined 1
```

Can I change the number of queues to something like 4, and the user space program,  have one AF_XDP socket per queue and one thread per AF_XDP socket, in order to have four threads processing traffic coming into the same NIC?

Thank you very much for your help and time.
Rio


From: Magnus Karlsson <magnus.karlsson@xxxxxxxxx>
Date: Saturday, December 10, 2022 at 6:57 AM
To: Zhaoxi Zhu <zzhu@xxxxxxxxxxxxx>
Subject: Re: Is It Possible to RX/Process/TX packets concurrently with AF_XDP?

No, that is not possible without expensive mutual exclusion mechanisms. Uae one socket per thread instead.

Magnus 
Le ven. 9 déc. 2022, 23:49, Zhaoxi Zhu <mailto:zzhu@xxxxxxxxxxxxx> a écrit :
Hi Magnust,

Thank you very much for your reply and the link you provided.

Do you think it is okay to have multiple threads for the same AF_XDP socket? In the AF_XDP-forwarding example, it seems like that the same AF_XDP socket is only handled by one thread. I wonder if that's okay for this AF_XDP-forward program to run like:

```
./xdp_fwd -i IFA -q QA -c CX -c CY
``` 

So that we have two threads running on the same AF_XDP socket.

Thank you again for your help.
Rio Zhu

On 12/8/22, 2:54 AM, "Magnus Karlsson" <mailto:magnus.karlsson@xxxxxxxxx> wrote:

    On Wed, Dec 7, 2022 at 11:54 PM Zhaoxi Zhu <mailto:zzhu@xxxxxxxxxxxxx> wrote:
    >
    > Hi All,
    >
    > Thank you very much for reading this email. My name is Rio.
    >
    > I recently started looking into the XDP technology, especially AF_XDP, and I really love it. I started studying and modifying this AF_XDP example(https://nam11.safelinks.protection.outlook.com/?url=https://github.com/xdp-project/xdp-tutorial/blob/master/advanced03-AF_XDP/af_xdp_user.c&data=05|01|zzhu@xxxxxxxxxxxxx|a48e497c567c44334e3508dadabed0f9|0fee8ff2a3b240189c753a1d5591fedc|1|0|638062810350112528|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0=|3000|||&sdata=odDqjtTb0xvO+iz1o5qd2DjPi0K51dtocSxwBA41Yf4=&reserved=0) to meet my need, and it has been working fine.
    >
    > However, one thing I notice is that this user space application is single threaded. I wonder if it is feasible to multi-threading to the RX/packet processing/TX parts of the program, in order to utilize other cores and possibly make my application faster?
    >

    Please check out the AF_XDP-forwarding example in this repo:

    https://nam11.safelinks.protection.outlook.com/?url=https://github.com/xdp-project/bpf-examples&data=05|01|zzhu@xxxxxxxxxxxxx|a48e497c567c44334e3508dadabed0f9|0fee8ff2a3b240189c753a1d5591fedc|1|0|638062810350112528|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0=|3000|||&sdata=hwn1indUjoqf4hkLrb94sWeVc96az7jnsc/GyMAA4pM=&reserved=0

    > One challenge I face now is, as I tried different places to add multi-threading, the program does not work properly. Symptoms such as `assert(xsk->umem_frame_free < NUM_FRAMES);` failed; ICMP ping packets doesn’t arrive destination until seconds later and TCP connection generated by `iperf` cannot be established occurs as I tried different places to add multi-threading to my code.
    >
    > So, my question is, does AF_XDP applications support multi-threading during RX/packet processing/TX? If so, what is a proper way to modify the AF_XDP example code to let it run properly?
    >
    > Thank you again for reading this email. I look forward to hearing from you.
    >
    > Best,
    > Rio Zhu
    >
    >





[Index of Archives]     [Linux Networking Development]     [Fedora Linux Users]     [Linux SCTP]     [DCCP]     [Gimp]     [Yosemite Campsites]

  Powered by Linux