2022-01-20 14:19 UTC+0000 ~ Dave Thaler <dthaler@xxxxxxxxxxxxx> > Quentin Monnet <quentin@xxxxxxxxxxxxx> wrote: >> Hi, I have the pleasure to announce the availability of a mirror for bpftool on GitHub, at the following URL: >> [...] >> 3. Another objective was to help other projects build on top of the existing sources for bpftool. I'm thinking in >> particular of eBPF-for-Windows, which has been working on a proof-of-concept port of the tool for Windows [1]. >> Bpftool's mirror keeps the minimal amount of necessary headers, and stripped most of them from the definitions >> that are not required in our context, which should make it easier to uncouple bpftool from Linux. >> [...] >> Just to make it clear, bpftool's mirror does not change the fact that all bpftool development happens on the >> kernel mailing-lists (in particular, the BPF mailing-list), and that the sources hosted in the kernel repository >> remain the reference for the tool. At this time the GitHub repository is just a mirror, and will not accept pull >> requests on bpftool's sources. > > Thanks Quentin, this is a great first step! I can update the ebpf-for-windows project to use this as a submodule. > > Longer term, is the goal to make the mirror be the authoritative reference, or to make the Linux kernel repository > not be Linux-only but accept non-Linux patches to bpftool? Hi Dave, longer term goals have not been established yet, and the discussion about what happens to bpftool next still needs to happen. I understand that you have been working on making bpftool cross-OS, and that this raises the question of how to contribute Windows-related patches upstream. Moving bpftool out of the kernel and into its own tree (whether on GitHub or on kernel.org) would make sense to me, although it comes with a number of things to sort out. First, bpftool is now being used directly by a number of components in the kernel, for loading programs or for its ability to generate BPF skeletons for programs. As far as I can tell, this concerns the following items: - The kernel itself, when configured with CONFIG_BPF_PRELOAD, requires bpftool to build, because BPF pre-loaded iterators rely on BPF skeletons (see kernel/bpf/preload/iterators/Makefile). - BPF samples and selftests (samples/bpf/Makefile, tools/bpf/runqslower/Makefile) use BPF for a number of use cases. - Other tools hosted in the kernel repository, in particular runqslower (tools/bpf/runqslower/Makefile) and perf (tools/perf/Makefile.perf), use bpftool to produce BPF skeletons as well. As far as I can tell, the above do not rely on cutting-edge bpftool features, and they could maybe be adjusted to consider bpftool as an external dependency for BPF, somewhat like pahole or clang/LLVM have been so far. Another thing to consider is that keeping bpftool next to the kernel sources has been useful to help keeping the tool in sync, for example for adding new type names to bpftool's lists when the kernel get new program/map types. We have recently introduced some CI checks that could be adjusted to work with an external repo and mitigate this issue, but still, it is harder to tell people to submit changes to a second repository when what they want is just to update the kernel. I fear this would result in a bit more maintenance on bpftool's side (but then bpftool's requirements in terms of maintenance are not that big when compared to bigger tools, and maybe some of it could be automated). Then the other solution, as you mentioned, would be to take Windows-related patches for bpftool in the Linux repo. For what it's worth, I don't have any personal objection to it, but it raises the problems of testing and ownership (who fixes bugs) for these patches. I'm also unsure what it would mean in terms of development workflow: would Windows-related contributions be reviewed and tested beforehand, and treated somewhat like vendor code, or would all the discussions (Windows-related bug reports, contributions to Windows support but external to Microsoft, etc.) happen on the BPF mailing list? If we want bpftool to become fully cross-OS, my feeling is that it would be maybe more work, but more sensible to move it outside of the kernel tree (although this does not have to be immediate, obviously - let's see how the Windows port is doing first). However, this decision is not mine alone to take, and the maintainers will surely have their say in it (this could also be a topic for you to raise at the next BSC meeting, I guess). I hope the considerations above can help for this discussion. Best regards, Quentin
