On Tue, Jan 14, 2025 at 01:29:11PM -0800, Song Liu wrote: > Hi Daniel, > > On Tue, Jan 14, 2025 at 1:02 PM Daniel Xu <dxu@xxxxxxxxx> wrote: > > > > Hi Song, > > > > On Mon, Jan 13, 2025 at 03:32:59PM -0800, Song Liu wrote: > > > On Fri, Jan 10, 2025 at 1:23 PM Daniel Xu <dxu@xxxxxxxxx> wrote: > [...] > > > > > > Maintaining out-of-tree kernel modules is a lot of work. I wonder whether > > > the benefit would justify this extra work. There are other ways to make > > > small changes to the built-in verifier, i.e. kernel live patch. > > > > The goal (in my mind) is not to maintain a full out-of-tree module. > > Rather, it'd be to do a 1-way sync out of the kernel and potentially > > apply some out-of-tree compatability patches. Same idea as libbpf: > > https://github.com/libbpf/libbpf. > > The idea can be practical if we can support the verifier with the same > model as libbpf. But I am not sure whether this is possible. Based on the research I've done, I believe it should be possible. I have some notes laying around but my plan is to start prototyping soon to prove this to myself. If there are specific concerns about why it'd be impossible, I'd appreciate hearing about it - better to find out early :) A separate point to consider would be if modular verifier can be done in a clean way. That would be on me to prove. > > > Verifier development should still happen in kernel tree. For folks who > > do not care about modular verifier, life should go on same as before. > > > > w.r.t. KLP, I'm not sure KLP satisfies the use case. For example, it > > seems unwieldy to potentially live-patch hundreds to thousands of > > patches. And since verifier is an algorithm heavy construct, we cannot > > get away from data structure changes -- IIUC something KLP is not good > > at. > > It is correct that it is only practical to make small changes with KLPs. But I > wonder how often we do need major changes to the verifier. The overarching goal is to deploy as many upstream changes as possible to older installed kernels. So I'd say there are a lot of major changes we'd way to deploy. For example, it would be reasonable (at this point in time) to want to deploy something like this diff stat: $ git --no-pager diff v6.6..HEAD --stat -- kernel/bpf/verifier.c kernel/bpf/verifier.c | 8907 ++++++++++++++++++++++++++++++++++--------------- 1 file changed, 6142 insertions(+), 2765 deletions(-) Wouldn't it also be problematic to have two changes modifying the same function? Seems like operational hassle to start merging patches. > > > > > > > > > > > > On top of delivering newer verifiers to older kernels, the facade opens the > > > > door to running the verifier in userspace. If the verifier becomes sufficiently > > > > portable, we can implement a userspace facade and plug the verifier in. A > > > > possible use case could be integrating the verifier into Clang [3] for tightly > > > > integrated verifier feedback. This would address a long running pain point with > > > > BPF development. This is a lot easier said than done, so consider this highly > > > > speculative. > > > > > > I think we don't need the verifier to be a LKM to do verification in user > > > space. Instead, we just need a mechanism to bypass (some logic of) > > > the verifier. Would this work? > > > > It's the other way around. The goal is not to _move_ verification into > > userspace but rather pre-verify. That way when the kernel verifies it > > you have a lot more confidence it will succeed. > > I think we had the pre-verify idea for quite some time. It will be > valuable if we can manage it without much extra effort. (Development > happens in the kernel, etc.) Agreed.
