On Wed, Jan 11, 2023 at 1:29 PM Song Liu <song@xxxxxxxxxx> wrote: > > () > > On Mon, Jan 9, 2023 at 9:47 AM Andrii Nakryiko > <andrii.nakryiko@xxxxxxxxx> wrote: > > > > On Thu, Jan 5, 2023 at 6:54 PM Alexei Starovoitov > > <alexei.starovoitov@xxxxxxxxx> wrote: > > > > > > On Thu, Jan 05, 2023 at 01:01:56PM -0800, Andrii Nakryiko wrote: > > > > Didn't find the best place to put this, so it will be here. I think it > > > > would be beneficial to discuss BPF helpers freeze in BPF office hours. > > > > So I took the liberty to put it up for next BPF office hours, 9am, Jan > > > > 12th 2022. I hope that some more people that have exposure to > > > > real-world BPF application and pains associated with all that could > > > > join the discussion, but obviously anyone is welcome as well, no > > > > matter which way they are leaning. > > > > > > > > Please consider joining, see details on Zoom meeting at [0] > > > > > > > > For the rest, please see below. I'll be out for a few days and won't > > > > be able to reply, my apologies. > > > > > > > > [0] https://docs.google.com/spreadsheets/d/1LfrDXZ9-fdhvPEp_LHkxAMYyxxpwBXjywWa0AejEveU/edit#gid=0 > > > > > > Thanks for adding it to the agenda. > > > Hopefully we'll be able to converge faster on a call. > > > > Yep, hopefully. Looking forward to BPF office hours this week. > > > > > > > > There are several things to discuss: > > > 1. whether or not to freeze helpers. > > > 2. whether dynptr accessors should be helpers or kfuncs. > > > 3. whether your future inline iterators should be helpers or kfuncs. > > > 4. whether cilium's bpf_sock_destroy should be helper or kfunc. > > I think these are all big questions. Maybe we can start with some > smaller questions? Here is a list of questions I have: > > 1. Do we want stable kfuncs (as stable as helpers)? Do we want > almost stable kfuncs? Yes. We've touched on some of that earlier. We can talk about a range: unstable, deprecated, starting to deprecate, stable plus orthogonal versioning scheme. > Will most users of stable APIs be as happy > with almost stable alternatives? kfuncs are very much analogous to EXPORT_SYMBOL_GPL. There is no versioning scheme, nor deprecation scheme for that. Yet in-kernel and out-of-tree users have been dealing with it. There are kABI things that make things stable to various degrees. So 'happy' is relative. Using that analogy... In-kernel bpf progs won't care. unstable or not they will get carried along automatically when kfuncs change. Out of tree bpf progs can be divided to kernel dependent and kernel independent. The former are similar to in-tree with extra pain that can be mitigated with kfunc detection. The latter will always use stable with understandable deprecation path. Yet it's all in theory. In practice networking folks are using conntrack kfuncs and xfrm kfuncs assuming we will make it all work somehow, though right now we're saying kfuncs are unstable only. So 'happy' and 'pain' are relative depending on the usefulness of kfunc. If bpf prog needs a feature it will use it. If it's a shiny new feature, the prog authors might wait until kfunc stabilizes. Which is exactly the point. We can wish for something to be useful, but we won't know until we actually use it for real and not in some selftest. And it becomes chicken and egg. If it's a cool new feature the bpf prog wants it to be stable to rely on it later, but because it's so new it's not clear whether it's actually useful, so we shouldn't be declaring it stable and cause kernel pains. > 2. Do we decide the stability of a kfunc when it is first added? Or > do we plan to promote (maybe also demote?) stability later? Claiming that something is stable on day one is a subjective opinion of the developer who's adding that feature. There could even be a giant user space project next to it attempting to use that feature, but we've seen that with other uapi-s in the past. > 3. Besides stability, what are the concerns with kfuncs? How hard > is it to resolve them? > AFAICT, the concerns are: require BTF, require trampoline. Only the former. kfuncs do not require bpf trampoline. $ git grep bpf_jit_supports_kfunc_call arch/arm64/net/bpf_jit_comp.c:bool bpf_jit_supports_kfunc_call(void) arch/loongarch/net/bpf_jit.c:bool bpf_jit_supports_kfunc_call(void) arch/x86/net/bpf_jit_comp.c:bool bpf_jit_supports_kfunc_call(void) arch/x86/net/bpf_jit_comp32.c:bool bpf_jit_supports_kfunc_call(void) iirc I've seen the patches for risc-v and arm32. > Anything else? I guess we will never remove BTF dependency. > Trampoline dependency is hard to resolve, but still possible? > > 4. We have feature-rich BPF with Linux-x86_64. Do we need some > bare-minimal BPF, say for Linux-MIPS, or Windows-ARM, or > even nvme-something? I guess this is also related to the BPF > standard? It's not related to ISA standardization. We're not even talking about BTF standardization. Nor about psABI (calling convention and such). It's going to happen much much later.
