On Tue, Dec 13, 2022 at 8:11 PM Zhaoxi Zhu <zzhu@xxxxxxxxxxxxx> wrote: > > It looks like that I didn’t include the mailing list in my previous replies. I hope this one does. > > Also, for the AF_XDP-forwarding example, is it able to handle multiple AF_XDP sockets on the same NIC? Yes. > Such as: > > ``` > ./xdp_fwd -i IFA -q Q1 -i IFA -q Q2 -i IFA -q Q3 -i IFA -q Q4 -c CX -c CY > > ``` > > If the above is doable, maybe I can have multiple queues, rather than having one, on the same NIC, create one AF_XDP socket per queue, and then use this xdp_fwd example to achieve multi-threading? That is the best way to do multithreading without having to resort to expensive locking. One queue and socket per thread is the way to go. > Thank you very much for your help and time. > Rio > > > From: Zhaoxi Zhu <zzhu@xxxxxxxxxxxxx> > Date: Monday, December 12, 2022 at 11:06 AM > To: Magnus Karlsson <magnus.karlsson@xxxxxxxxx> > Subject: Re: Is It Possible to RX/Process/TX packets concurrently with AF_XDP? > > Got it, thank you very much for your clarification. > > I have one more question, if I may: If one AF_XDP should be handled by one thread, in order to avoid mutexes and to achieve better performance, then, can I have more than one AF_XDP socket on the same physical NIC, and use one thread per AF_XDP socket, in order to make process packets coming into this NIC concurrently? > > Currently, the way we are testing AF_XDP with is to have only 1 queue: > > ``` > sudo ethtool -L <interface> combined 1 > ``` > > Can I change the number of queues to something like 4, and the user space program, have one AF_XDP socket per queue and one thread per AF_XDP socket, in order to have four threads processing traffic coming into the same NIC? > > Thank you very much for your help and time. > Rio > > > From: Magnus Karlsson <magnus.karlsson@xxxxxxxxx> > Date: Saturday, December 10, 2022 at 6:57 AM > To: Zhaoxi Zhu <zzhu@xxxxxxxxxxxxx> > Subject: Re: Is It Possible to RX/Process/TX packets concurrently with AF_XDP? > > No, that is not possible without expensive mutual exclusion mechanisms. Uae one socket per thread instead. > > Magnus > Le ven. 9 déc. 2022, 23:49, Zhaoxi Zhu <mailto:zzhu@xxxxxxxxxxxxx> a écrit : > Hi Magnust, > > Thank you very much for your reply and the link you provided. > > Do you think it is okay to have multiple threads for the same AF_XDP socket? In the AF_XDP-forwarding example, it seems like that the same AF_XDP socket is only handled by one thread. I wonder if that's okay for this AF_XDP-forward program to run like: > > ``` > ./xdp_fwd -i IFA -q QA -c CX -c CY > ``` > > So that we have two threads running on the same AF_XDP socket. > > Thank you again for your help. > Rio Zhu > > On 12/8/22, 2:54 AM, "Magnus Karlsson" <mailto:magnus.karlsson@xxxxxxxxx> wrote: > > On Wed, Dec 7, 2022 at 11:54 PM Zhaoxi Zhu <mailto:zzhu@xxxxxxxxxxxxx> wrote: > > > > Hi All, > > > > Thank you very much for reading this email. My name is Rio. > > > > I recently started looking into the XDP technology, especially AF_XDP, and I really love it. I started studying and modifying this AF_XDP example(https://nam11.safelinks.protection.outlook.com/?url=https://github.com/xdp-project/xdp-tutorial/blob/master/advanced03-AF_XDP/af_xdp_user.c&data=05|01|zzhu@xxxxxxxxxxxxx|a48e497c567c44334e3508dadabed0f9|0fee8ff2a3b240189c753a1d5591fedc|1|0|638062810350112528|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0=|3000|||&sdata=odDqjtTb0xvO+iz1o5qd2DjPi0K51dtocSxwBA41Yf4=&reserved=0) to meet my need, and it has been working fine. > > > > However, one thing I notice is that this user space application is single threaded. I wonder if it is feasible to multi-threading to the RX/packet processing/TX parts of the program, in order to utilize other cores and possibly make my application faster? > > > > Please check out the AF_XDP-forwarding example in this repo: > > https://nam11.safelinks.protection.outlook.com/?url=https://github.com/xdp-project/bpf-examples&data=05|01|zzhu@xxxxxxxxxxxxx|a48e497c567c44334e3508dadabed0f9|0fee8ff2a3b240189c753a1d5591fedc|1|0|638062810350112528|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0=|3000|||&sdata=hwn1indUjoqf4hkLrb94sWeVc96az7jnsc/GyMAA4pM=&reserved=0 > > > One challenge I face now is, as I tried different places to add multi-threading, the program does not work properly. Symptoms such as `assert(xsk->umem_frame_free < NUM_FRAMES);` failed; ICMP ping packets doesn’t arrive destination until seconds later and TCP connection generated by `iperf` cannot be established occurs as I tried different places to add multi-threading to my code. > > > > So, my question is, does AF_XDP applications support multi-threading during RX/packet processing/TX? If so, what is a proper way to modify the AF_XDP example code to let it run properly? > > > > Thank you again for reading this email. I look forward to hearing from you. > > > > Best, > > Rio Zhu > > > > >
|