Hi friends of XDP,
I wrote a small ebpf code generator for ebpf. The idea is that one
writes what mostly looks like a Python program, yet it actually
generates ebpf. The advantage is that everything is done in one go, no
external compilation is needed.
To give an example, the following is the complete code needed to
simply count incoming packages. Note that the code does define a
HashMap, which can be accessed both from XDP and user space. The
reading and writing of the hash map is done transparently by the
library:
from asyncio import get_event_loop, sleep
from ebpfcat.hashmap import HashMap
from ebpfcat.xdp import XDP, XDPExitCode
class Count(XDP):
license = "GPL"
userspace = HashMap()
count = userspace.globalVar()
def program(self):
# this code actually generates ebpf:
self.count += 1
self.exit(XDPExitCode.PASS)
async def main():
c = Count()
await c.attach("eth0")
for i in range(100):
await sleep(0.1)
# c.count can be accessed from userspace!
print("packets arrived so far:", c.count)
if __name__ == "__main__":
get_event_loop().run_until_complete(main())
Everything can be found on github: https://github.com/tecki/ebpfcat
PyPI: https://pypi.org/project/ebpfcat/ and readthedocs:
https://ebpfcat.readthedocs.io/en/latest/
The code is still in its infancy, but the above example already works.
I hope that this contribution will make XDP development much easier,
expecially for newbies (like me...)
To give you some background why I did this, which is most likely
completely unrelated to what others do on this list, but for your
entertainment:
I want to control hardware using the EtherCAT protocol. This protocol
is directly on top of EtherNet, and basically what you have to do is
to write out an EtherNet packet to the hardware to tell it to do
something, and it returns a packet with the changes that this resulted
in, ending in a ping-pong between the hardware and the controlling
computer. In this scenario, latency is everything: the faster you can
react to packets coming in, the better is the control of the hardware.
XDP promises good latency, so I gave it a shot. The reason I wrote a
Python code generator is that I need to generate a dedicated XDP
program for the connected hardware. Usually one writes a generic
library that can deal with all kinds of hardware, but given that ebpf
programs are pretty limited in size, I have to generate code
on-the-fly after having detected the current hardware configuration.
Now to the results: on my Raspberry Pi 4 I was able to process 30 000
packets/s. This probably is a good laugh for most on this list, but
for EtherCat this is pretty amazing, as this is close to the hardware
limit. Interestingly, when I used my laptop, which has an Intel
ethernet card that uses the e1000e kernel driver, I was only able to
process 10 000 packets/s, even though I had switched off all buffering
using ethtools. Worse, sometimes (I guess around one out of a 100 000
packets) the packet got stuck somewhere for more than a millisecond.
This is awful for my application, and never happened on my Raspberry
Pi 4. In the end I don't care much as a Raspi is pretty cheap, and if
it does the job, fine.
But this still does puzzle me. Why does the e1000e sometimes delay
packets? If the e1000e driver was XDP enabled (I read that somebody
seems to be working on that), would that improve the situation? And
what about my Raspi, could I improve the situation even more by adding
XDP code to its network driver?
Hope you have fun with my ideas
Martin
|