On Wed, Feb 17, 2021 at 9:32 AM Jonathan Corbet <corbet@xxxxxxx> wrote: > > [CC += linux-doc] > > Joe Stringer <joe@xxxxxxxxxxx> writes: > > > From: Joe Stringer <joe@xxxxxxxxx> > > > > The state of bpf(2) manual pages today is not exactly ideal. For the > > most part, it was written several years ago and has not kept up with the > > pace of development in the kernel tree. For instance, out of a total of > > ~35 commands to the BPF syscall available today, when I pull the > > kernel-man-pages tree today I find just 6 documented commands: The very > > basics of map interaction and program load. > > > > In contrast, looking at bpf-helpers(7), I am able today to run one > > command[0] to fetch API documentation of the very latest eBPF helpers > > that have been added to the kernel. This documentation is up to date > > because kernel maintainers enforce documenting the APIs as part of > > the feature submission process. As far as I can tell, we rely on manual > > synchronization from the kernel tree to the kernel-man-pages tree to > > distribute these more widely, so all locations may not be completely up > > to date. That said, the documentation does in fact exist in the first > > place which is a major initial hurdle to overcome. > > > > Given the relative success of the process around bpf-helpers(7) to > > encourage developers to document their user-facing changes, in this > > patch series I explore applying this technique to bpf(2) as well. > > So I am totally in favor of improving the BPF docs, this is great work. > > That said, I am a bit less thrilled about creating a new, parallel > documentation-build system in the kernel. I don't think that BPF is so > special that it needs to do its own thing here. > > If you started that way, you'd get the whole existing build system for > free. You would also have started down a path that could, some bright > shining day, lead to this kind of documentation for *all* of our system > calls. That would be a huge improvement in how we do things. > > The troff output would still need implementation, but we'd like to have > that anyway. We used to create man pages for internal kernel APIs; that > was lost in the sphinx transition and hasn't been a priority since > people haven't been screaming, but it could still be nice to have it > back. > > So...could I ask you to have a look at doing this within the kernel's > docs system instead of in addition to it? Even if it means digging into > scripts/kernel-doc, which isn't all that high on my list of Fun Things > To Do either? I'm willing to try to help, and maybe we can get some > other assistance too - I'm ever the optimist. Hey Jon, thanks for the feedback. Absolutely, what you say makes sense. The intent here wasn't to come up with something new. Based on your prompt from this email (and a quick look at your KR '19 presentation), I'm hearing a few observations: * Storing the documentation in the code next to the things that contributors edit is a reasonable approach to documentation of this kind. * This series currently proposes adding some new Makefile infrastructure. However, good use of the "kernel-doc" sphinx directive + "DOC: " incantations in the header should be able to achieve the same without adding such dedicated build system logic to the tree. * The changes in patch 16 here extended Documentation/bpf/index.rst, but to assist in improving the overall kernel documentation organisation / hierarchy, you would prefer to instead introduce a dedicated Documentation/userspace-api/bpf/ directory where the bpf uAPI portions can be documented. >From the above, there's a couple of clear actionable items I can look into for a series v2 which should tidy things up. In addition to this, today the bpf helpers documentation is built through the bpftool build process as well as the runtime bpf selftests, mostly as a way to ensure that the API documentation conforms to a particular style, which then assists with the generation of ReStructured Text and troff output. I can probably simplify the make infrastructure involved in triggering the bpf docs build for bpf subsystem developers and maintainers. I think there's likely still interest from bpf folks to keep that particular dependency in the selftests like today and even extend it to include this new Documentation, so that we don't either introduce text that fails against the parser or in some other way break the parser. Whether that validation is done by scripts/kernel-doc or scripts/bpf_helpers_doc.py doesn't make a big difference to me, other than I have zero experience with Perl. My first impressions are that the bpf_helpers_doc.py is providing stricter formatting requirements than what "DOC: " + kernel-doc would provide, so my baseline inclination would be to keep those patches to enhance that script and use that for the validation side (help developers with stronger linting feedback), then use kernel-doc for the actual html docs generation side, which would help to satisfy your concern around duplication of the documentation build systems. Cheers, Joe
