Notes from eBPF Side Meeting @ IETF 115 Note Takers: . Mykola Lysenko mailto:mykolal@xxxxxxxx (remote) . Kiran Makhijani mailto:kiran.ietf@xxxxxxxxx (in room) Recording: https://www.youtube.com/watch?v=KRGYGBkJpWM Attendance: . Includes contributors and leaders from both the eBPF community and the IETF community . 17 on zoom o Alexei Starovoitov (BPF Steering Committee) o Daniel Migault o Gyan Mishra o KP Singh (BPF Steering Committee) o Manu o Mykola Lysenko o Quentin Monnet o Sharon Z. o . (others) . More than 16 in room o Christoph Hellwig o Dave Thaler (BPF Steering Committee) o David Black o Erik Kline (INT Area Director) o Lars Eggert (IETF Chair) o Lorenzo Colitti o Kiran Makhijani o Mirja Kühlewind (IAB Chair) o Paul o Suresh Krishnan o . (others) -------------------------------------------------------- Meeting started at 6:05 PM with introductory slides by Dave -------------------------------------------------------- The goal of this meeting was to discuss where to publish eBPF specification(s), where IETF may or may not be the right answer. The goal is not to go in technical options (not a tech meeting). Option A: publishing through BPF foundation . Historically all bpf documents are in the Linux kernel source tree. They get reviewed and merged. Recently, a Github mirror under eBPF Foundation was created. We are in the middle of the process to generate specification as pdf from the docs in the Linux kernel tree. Option B. ISO/IEC JTC1 (after doing A) . e.g. are W3C and OCF (links in the slides). . Q: Are these ISO documents freely available. . A: Yes . documents can be updated, obsoleted by the newer versions of it . may take up to 17 months to publish . not super fast. They have meetings every 6 months. Long process; however, some org. provide it for free. . David Black: They don't change the document name/title but the year of publication. Option C: Independent Submission Stream RFC . submission stream RFCs outside of the IETF, but they would not be "standards" and no one was in favor of that approach. Option D: IETF stream RFC . Dave went through IETF process/terms for non-IETFers, based on the IETF Tao https://www.ietf.org/about/participate/tao/ ; . IETF and document authors own the copyright . Contributors and participants agree to disclose if they are personally aware of patents . IETF takes no position on patents themselves . Documents themselves do not mention patents 7 candidates for standardization were shared from previous BPF discussions in BPF office hours, Linux Plumbers, etc: . ISA - somewhat analogous to Java VM . ELF file format . BPF type format . Compiler expectations . Verifier expectations . Cross-platform map types (array, hash table, prefix table, etc) - some networking specific, some not . Cross-platform program types - some networking, some not Questions to answer: . Is any of this in scope for IETF? . Are IETF copyright/IPR rules ok? . Is the speed of publication ok? . Others? Today the BPF community currently submits changes to BPF mailing list for review We have a github repository synced with upstream Linux Kernel Dave shared the patch workflow from his slides So, is BPF in scope for IETF? Dave gave examples of cases where documents not directly related to routing/networking were published as IETF RFCs. Those are not just API but pseudo code in them. - Netlink specification is an RFC (informational). https://www.rfc-editor.org/info/rfc3549 - JavaScript is an RFC but also published by ECMA - Some socket API extensions are in an RFC (e.g., https://www.rfc-editor.org/info/rfc3678) but it is informational not a standard, the standard comes from POSIX - Java VM would be similar analogy but is not in any RFC - A file format could be in scope of IETF (e.g., https://www.rfc-editor.org/info/rfc9116) but that example was for a format inherently tied to networking. ELF file format isn't inherently tied to networking though. - Tao of IETF recommends BOF to start a WG, goals for BoF include determining whether a critical mass of IETF participants exists, and that IETF is the right place for that activity. -------------------------------------------------------- Open discussion then started at 6:22 PM -------------------------------------------------------- Lars: Dave and I talked about it at the Linux conference in Dublin. QUIC we did option A. . thinks it is in scope of IETF . if it is not of scope for IETF, we should change the scope . github approach would work for us [Someone on the Zoom] Are you open to standardize an interface? Do you mean relationship between the program-type and network specifics like (unicast/multicast). <--can not hear --> KP: ?? Erik: At the minimum we have ART area and it fits under that area. Lars: we will find an area. Dave: there are use cases that fit in both INT and ART area. Different cases will have diff areas. Analogy is where should DHCP option go? Answer is that today a DHCP option can be done in the WG for whatever the option is configuring, with review by the DHC WG. Sharon: . question: are you open to standardizing the interface to solve the problem using the BPF set of skills? it is not for the sake of eBPF, but for the . . examples: people porting from cloud env to edge env, no streams that edge graged, we can combine bpf and streams to help port streams to the edge Dave: . networking specific, interfaces . Sharon might have spoken about networking specific program types Sharon: . Kafka does not work in the edge . stream would be a topic . it would leverage BPF . it would be useful to have kafka like streams KP: . eBPF code will run on routing equipment itself? Sharon: . no, it will run on Linux . it will be an object portable by cilium . it will have logical address associate with it . city has a set of engines . off at night . vehicles will send packets of their instrumentation Eric: . gets too technical . can we get back to the less technical conversation Sharon: . specific example of the networking . network management area o metric constructs o subnet conversations . the question are you open to specifying the RFC that is not specific to BPF but using BPF extensively Mirja: To clarify on process, pdf is the only documentation? Dave: yes, just to prove the process works. Mirja: clarification question - so you need to standardize existing documents. . I do not understand the technical topic enough . Is this the only documentation we have for now? Lorenzo: you need to standardize when interoperability is important and for backward compatibility. backward compatibility implications will have to be provided by BPF. Is it like a Linux guarantee, e.g. never breaking userspace? Some APIs are guaranteed forever... some other things evolve more quickly. Mykola: ISA is not stable David: We can definitely find a place here... <missed some part...> Suresh: . (1) routing is a better area. Netlink and TIPC are another examples that group worked on. . (2) eBPF community needs to think about change control. Anyone from IETF can have a discussion with you on changes, you will need to do the work here based on IETF processes. . (3) in terms of updates and backward incompatible changes - 4 in 6yrs are good, but not 4 in 6 months. Dave replying to Suresh: Some types of changes can break backward compatibility, such as in Instruction set architecture (ISA) are rare (gave example about divide-by-0 behavior). Whereas addition of new instructions is not breaking backward compatibility. - As long as 'sufficient' Linux community is participating, the risk is low and manageable. . two types of modifications done to ISA o things can get from undefined to defined, e.g., what do you do if you try to divide by a 0? now it is acceptable, before it was a broken program o new instructions were added . as long as Linux kernel community participates in the decision making - the risk is low two communities will diverge Christoph: . (1) divide by 0 change was a good example where something that was undefined got defined and behavior changes. The behavior was not documented, that was the other thing. . (2) it is surprising how the 2 community works are similar, if Linux maintainer doesn't take it the functionality does not go to kernel. Eric Kline: FoRCeS did netlink. You dont have to wait for full publication before you implement it. . we can take a document and go through the process . if Linus does not agree then you do not have consensus Paul: understand the change control clearly, IETF has rough consensus, in Linux if Linus doesn't like it he vetoes it, that's not how IETF does it. Lorenzo: Question for eBPF - is there a canary document, something that is available and can be brought to IETF? e.g., HTTP3 was standardized later in IETF, was brought by Google. Dave followed up with examples of those (ISA, file format etc. from the slides) Mykola, remote: why do we need to do standard? Christoph: because some hardware vendors have aspiration to see and verify against a standard specification. gave NVMe example, when you download code for storage h/w, people ask for standard document. NVMe needs to have an ability to have one standard document reference, we really like IETF specs Lorenzo: We need standards when more than one vendor is involved. Lucas reply to Mykola: initiative is good. Will help get better collaboration . BPF networking code . applications want to use <<someone>> what is IETF going to offer? -- You get RFC label, some of the work is not properly labeled. it will add an overhead, we need to document existing practices Dave: right now we are documenting existing practice Gyan: eBPF can help with programmability of routing protocols as reference open platform. will help in interoperability on different hardware: . Would eBPF programmability be ported to any hardware? . Is there a desire to port to router platforms? . if yes, that would be a reason for interoperability . BPF is programmability . P4 programming, ASIC programming, beneficial for optimizations . Does it really come into play? Christoph: both for being covered Lorenzo: move on reference interop platforms - it will be like support on cisco/juniper or kernel. Non-Linux eBPF benefits are extremely useful. we use bpf in android Christoph: . interop . another huge advantage is having people asking the questions Eric Kline: recommending to bring this to RTGAREA. on programmability, eBPF can improve the readability of the IETF documents. SRv6 will be a good example. eBPF was used a lot to implement protocols in hackathon. you cannot process some options in the HW Dave: eBPF was in multiple hackathon presentations last weekend Alexei: two interesting things . standardize ISA to standardize other things . other parts like routing, maybe to consider standardizing using BPF Christoph said he was willing take the lead on dealing with IETF process and overheads for the documentation. mailing list: mailto:bpf@xxxxxxxxxxxxxxx ----------------------------------------- Kiran's summary: 1. Unanimous support from the IETF to eBPF. eBPF really keen on putting RFC # of ISA, file format etc. 2. Discussion on understanding what and how to standardize (it seems documents will be ready in kernel source, brought to IETF for standardization). If IETF review causes changes to an implementation - will it change existing implementation or not was not clear to me). 3. The eBPF community was warned about the overheads related to bringing work to IETF - rough consensus, process overheads, IPR, anybody could comment, etc. 4. Benefits are clear: interop. eBPF can share standard RFC to refer to for different vendor implementations (incl. hardware). 5. Implications of change control and backward incompatibility were discussed as potential risk and should be evaluated more seriously if eBPF is ok with slow-paced changes. 6. IETF community thinks it can use eBPF to improve documentation of its own protocols. e.g. ,SRV6 but how is not hashed out. Not the intent of eBPF. 7. Erik as Internet AD, said he wants this WG in routing area. Lars as IETF chair, supported 100%. 8. someone from eBPF took lead responsibility to bridge eBPF and IETF and mitigate risks discussed. ----------------------------------------- Mykola's summary: we went through Dave Thaler's slides and introduced the audience to the current BPF process of BPF documentation. Then we tried to answer the question if IETF is the right approach to do BPF standardization. Participants discussed a few existing RFC standards that are not networking related and have separate goals. E.g., compression, API interfaces, and others. We did not get a clear answer on why the hardware community needs to publish RFC except that they will be able to reference a particular version for the implementation and that IETF has a process reviewing such documents. In particular it is not clear why a versioned document published by the BPF community would not suffice. In the end, everyone agreed more discussion is required.
