On 12/14/22 16:31, Magnus Karlsson wrote:
On Wed, Dec 14, 2022 at 4:16 PM Federico Parola <federico.parola@xxxxxxxxx> wrote:On 12/14/22 10:00, Magnus Karlsson wrote:On Wed, Dec 14, 2022 at 9:16 AM Federico Parola <federico.parola@xxxxxxxxx> wrote:Hello everyone, in a context in which all traffic is sent to user space, and the main concern is performance (so AF_XDP busy polling is the best candidate), is there a reason to choose AF_XDP instead of DPDK for packet I/O, given that the latter is still much faster?The choice is not AF_XDP vs DPDK, since DPDK can run on top of AF_XDP. The choice is do you want to use user-space drivers or kernel space drivers. There are many pros and cons with both approaches. The two main advantages of user-space drivers in DPDK are that they are faster than their kernel counterparts and you get access to many NIC features that are not available right now from user-space if you are using a kernel driver. Kernel drivers, on the other hand, enables you to deploy your application on any Linux system (from a certain version) and does not force you into the strict execution model of DPDK (which is what most people use with user-space drivers). Everything in Linux is still usable and it is easy to have a system that mixes data-plane processing with other types of workloads. If the only thing you care about is max performance of a pure packet processing workload on a system you have complete control over, the choice is easy: go with a user-space driver and DPDK. If this is not your ultimate goal, then XDP and AF_XDP might be something to consider.Thanks for your answer Magnus. - Concerning deploying an application on any Linux system, what could be the limitations of DPDK userspace drivers that AF_XDP doesn't have? The use of hugepages?That would be one. The requirement of shared memory, forced unity mmap() calls that might fail, lack of binary forward compatibility, etc.- Concerning a system that mixes data-plane processing with other types of workloads, do you refer only to the capability of XDP to steer traffic either towards userspace with AF_XDP or let it proceed to the kernel? Or are there other features of DPDK userspace drivers that would impact this kind of system? Maybe the use of busy polling is too aggressive to share the CPU with other workloads?Yes, busy-polling and sharing a core is not a good idea in any system. But there is a good solution to this in DPDK that uses capabilities in newer CPUs. Basically, if you use DPDK you should adhere to its strict model of one process per core, otherwise you will likely get into trouble. Either by performance dropping significantly because you enable things like interrupts or 4K pages in DPDK, or because you do things like trying to run to DPDK polling threads on the same core, which might result in deadlock. If I am not mistaken, there is a concept in DPDK called service cores in which you can share a core between a normal process and a DPDK data plane thread. But it requires you to program specifically for it. If you want to steer part of the traffic to the kernel, you are better off using XDP. Bouncing things through user-space is expensive, DPDK or not.There are likely some good write-ups on the Internet about this. /MagnusBest regards, Federico Parola
Thank you very much for this information Magnus. On 12/14/22 20:46, Andrey Slastenov wrote:> Hello everyone > > Check out this short article about why XDP was chosen over DPDK. The > reasoning for this decision is also explained. >> https://gcore.com/blog/how-we-use-regular-expressions-in-xdp-for-packet-filtering/
> > > BR, Andrey Thanks for sharing this article Andrey. Best regards, Federico
|