On Thursday 13 September 2007 3:11:24 am Michael Kerrisk wrote: > Hey Rob, > > I just got myself subscribed to linux-doc, finally. Since I > do stuff in the nearby space (kernel-userland API docs), and > I haven't been following your endeavours closely on LKML, > I'm curious how things are going overall with kernel > documentation. Pretty well. I just got a skeleton of the master index up at http://kernel.org/doc/master.html (which is converted from http://kernel.org/doc/master.idx via make/indexsections.py). I'm not really happy with the section headers that the index links to. I just added section numbers which is makes it less bad but I'm still not quite comfortable with it. Maybe I need <hr> tags in there or something. Possibly they should be <h1>. I need to look at it in Firefox rather than just Konqueror... Currently I'm taking a break from that and working out a plan of attack for the EXPORT_SYMBOL macros in the source. There are over 12,000 of them, and they should all be documented. Probably as a "make htmldocs" book. I have an email about that which I sent to my Linux Foundation boss off-list, which I should probably dig up and repost here. > What are your rough master plans and projects going > forward, in terms of kernel documentation? (I ask > somewhat naively, because I'm not exactly sure what your > remit is for the LF fellowship year.) What's a "remit"? Do you mean my expected output? What I'm trying to do is: A) Get everything documentation-related from the kernel tarball onto the web where Google can find it. This includes the Documentation directory, "make htmldocs", the README files scattered in the source code, the menuconfig help, and miscelaneous like the output of "make help". Currently, all of that _is_ on the web, and updated by scripts I run when I remember to. (I'll probably make a cron job out of this eventually, but I don't want to run anything cpu-intensive on kernel.org and any use of "xmlto" falls deeply in that category.) B) Index the great heaps of documentation out there on the web, to create a central go-to site for people who want to know things about the Linux kernel. This includes lots of stuff that _won't_ be merged into the kernel tarball. There was some discussion about making the Documentation/* directory the central repository of all kernel knowledge and wisdom, but this doesn't work. Ignoring the fact that it's currently totally disorganized (which can be fixed), and also ignoring the fact that it's not even the central repository for in-kernel documentation (there are at least three other major sources of documentation in the kernel tarball listed above), there's a design problem. The Documentation directory is full of _text_files_, but lots of documentation is out on the web. Yes, there are good text files that go by (often on linux-kernel as patch [0/##], or as git checkin comments, or in developer blogs...) But there are also Linux Weekly News articles in html, OLS papers in PDF format, audio files in mp3 and ogg, video files in whatever Google puts its Tech Talks in (flash video, I think)... There are literally gigabytes of this stuff out there, in varying formats under varying licenses. It isn't even feasible to mirror all of it on kernel.org, let alone merge it into the kernel tarball, let alone convert it all to ascii text. If you're linking out to stuff on the web, the format you want to use is html, not text. Converting Documentation to html would be a big controversial change, and I'm Not Going There (tm). Documentation's job is to host documentation, but _my_ job is primarily editorial. I find it, link to it, and clean it up/fill in the holes as I have time. So I'm creating a master index of kernel documentation. Some of it will be local content, some of it will be mirrored locally, and some of it the index just links to and runs a 404 checker against (with reliance on the wayback machine for stuff we don't mirror until it goes down). > I'm not sure whether it's in the remit, or whether you have > the time or interest in helping out in kernel-userland > documentation (i.e., man-pages), but there are certainly > things to do there if you want. Let me know if you are > interested. The interface between kernel and userland is actually my main area of interest. There's not much point in documenting the stuff that's going away again in three releases, which includes most of the deep kernel internals. It's _nice_ having that stuff documented, but not the world's best use of time trying to nail jello to the wall. The kernel-userland interface is _stable_, so once we document that it STAYS DOCUMENTED (more or less), so there's no much point going after the more ephemeral stuff until the stable stuff's nailed down. The question was raised at the kernel summit: is man-pages documenting what the kernel exports, or is it documenting the interface glibc exposes to userspace (often through a heavy wrapper)? http://lwn.net/Articles/248376/ http://lwn.net/SubscriberLink/248376/24c8808f1c680d5e/ > Part of Michael's proposal is that new system calls should come equipped > with manual pages. It was suggested that this requirement will be hard to > enforce unless the man pages are packaged with the kernel itself. That led > to an interesting question: the man pages, as currently written, document > the system call interface as presented by the C library. But the API > exported directly by the kernel can be different, and often is. Which API > should be documented? It seems that the kernel-implemented API is the one > to cover, especially considering that glibc is not the only C library and > that other library implementors may well be very interested in that > information. Once again, "distribute it with the kernel" is seen as some kind of magic. It's not. Honest and truly it's not. It just means that updating the documentation results in your post getting lost in the noise on linux-kernel and having to go through the four-layer bureaucracy (developer, maintainer, sour cream, lieutenant, baked beans, Andrew, Linus) with somebody objecting along the way that your patch has the signed-off-by line in the wrong position and you forgot to include three magic dashes as mentioned section 14 paragraph 21 of Documentation/SubmittingPatches (no seriously, look it up), and in any case you weren't supposed to send it to THIS list you should have sent it to netfilter-devel (didn't you read http://vger.kernel.org/vger-lists.html? What's wrong with you?) and then it's still being ignored after the third repost two months later... And of course the stuff that's in the kernel now is such a marevlous example of up-to-dateness. Documentation/scsi/scsi.txt is primarily a link to two online documents (http://www.tldp.org/HOWTO/SCSI-2.4-HOWTO and http://www.torque.net/scsi/SCSI-2.4-HOWTO) which were each last updated about 5 years ago and are for the 2.4 kernel, yet despite that are the most up-to-date and comprehensive documentation available about a major kernel subsystem. (Yes, I asked.) I'd like to improve this, and plan to, but I'm not a scsi guy. I have to learn it before I can document it, along with everything else I have to do. Anyway, yes I am interested in helping out here, it's a question of free time. :) You may notice that I've kept up the Doclifter html translation of the man pages (http://kernel.org/doc/xmlman) since http://kernel.org/doc/man-pages remains empty. :) Among other things, I'm interested in documenting system calls and ioctls. I didn't even know "man 2 syscalls" existed until last week. (I was pulling up man 2 syscall and not getting an index. :) There isn't a man page for ioctls but I can start with: find /usr/include/linux -type f | xargs grep "_IO.(" Part of what I'd like to do is index the existing man pages more coherently. There's lots of great stuff there I just haven't had time to _read_. But then that's true about every decent source of documentation, you'll notice I have a large list of sources already. They need to be _integrated_, and that's serious work... > Best regards, > > Michael Did I answer your question? :) Rob -- "One of my most productive days was throwing away 1000 lines of code." - Ken Thompson. - To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
