Hi Song, On Mon, Jan 13, 2025 at 03:32:59PM -0800, Song Liu wrote: > On Fri, Jan 10, 2025 at 1:23 PM Daniel Xu <dxu@xxxxxxxxx> wrote: > > > > Hi folks, > > > > I'd like to propose modular BPF verifier as a discussion topic. > > Interesting idea! > > I was thinking about "modular verifier" these days, but in a different > way, i.e., how to divide the verifier into smaller modules (logical > modules, not necessarily kernel modules). We currently support > loading kfuncs in kernel modules, so it is natural to put the checks > of kfuncs into modules (via btf_kfunc_id_set.filter function). > > > === Motivation === > > > > A decade of production experience with BPF has shown that the desire for > > feature availability outpaces the ability to deliver new kernels into the field > > [0]. Therefore, the idea of modularizing the BPF subsystem into a loadable > > kernel module (LKM) has started to look appealing, as this would allow loading > > newer versions of the BPF subsystem onto older versions of the kernel without a > > reboot. > > > > [...] > > > > > For forward compatibility, the idea is to implement a facade built into each > > kernel that exposes a stable-enough (non-UAPI) interface such that the verifier > > can remain portable and “plug in” to the running kernel. While I expect the > > facade to be necessary, it will not be sufficient. There will eventually be > > details the facade cannot hide, for example an unavoidable ABI break. To solve > > for this, I/we [2] will maintain a continuously exported copy of the verifier > > code in a separate repository. From there we can develop branching, patching, > > or backport strategies to mitigate breaks. The exact details are TBD and will > > become more clear as work progresses. > > Maintaining out-of-tree kernel modules is a lot of work. I wonder whether > the benefit would justify this extra work. There are other ways to make > small changes to the built-in verifier, i.e. kernel live patch. The goal (in my mind) is not to maintain a full out-of-tree module. Rather, it'd be to do a 1-way sync out of the kernel and potentially apply some out-of-tree compatability patches. Same idea as libbpf: https://github.com/libbpf/libbpf. Verifier development should still happen in kernel tree. For folks who do not care about modular verifier, life should go on same as before. w.r.t. KLP, I'm not sure KLP satisfies the use case. For example, it seems unwieldy to potentially live-patch hundreds to thousands of patches. And since verifier is an algorithm heavy construct, we cannot get away from data structure changes -- IIUC something KLP is not good at. > > > > > On top of delivering newer verifiers to older kernels, the facade opens the > > door to running the verifier in userspace. If the verifier becomes sufficiently > > portable, we can implement a userspace facade and plug the verifier in. A > > possible use case could be integrating the verifier into Clang [3] for tightly > > integrated verifier feedback. This would address a long running pain point with > > BPF development. This is a lot easier said than done, so consider this highly > > speculative. > > I think we don't need the verifier to be a LKM to do verification in user > space. Instead, we just need a mechanism to bypass (some logic of) > the verifier. Would this work? It's the other way around. The goal is not to _move_ verification into userspace but rather pre-verify. That way when the kernel verifies it you have a lot more confidence it will succeed. Thanks, Daniel
