Hi Markus, On 04/08/16 17:12, Markus Heiser wrote: > Hi kernel-doc authors, > > motivated by this MT, I implemented a toolchain to migrate the kernel’s > DocBook XML documentation to reST markup. > > It converts 99% of the docs well ... to gain an impression how > kernel-docs could benefit from, visit my sphkerneldoc project page > on github: > > http://return42.github.io/sphkerneldoc/ > > The sources available at: > > https://github.com/return42/sphkerneldoc > > The work is underway, suggestions are welcome! I have to admit that this looks pretty good :-) My main remark based on my quick scan through the docs is that anything in typewriter font seems to be shown as red text with a rectangle around it. That's quite jarring for me and I think it should be just shown as normal text, just using a non-proportional font, just like in the original. I also noticed that the 'title' of tables ends with a '¶' character for some reason. See e.g. the struct v4l2_audioout table in http://return42.github.io/sphkerneldoc/books/linux_tv/media/v4l/vidioc-g-audioout.html Regards, Hans > > .. have a nice weekend .. > > --M-- > > > Am 13.03.2016 um 16:33 schrieb Markus Heiser <markus.heiser@xxxxxxxxxxx>: > >> >> Am 10.03.2016 um 16:21 schrieb Mauro Carvalho Chehab <mchehab@xxxxxxxxxxxxxxx>: >> >>> Em Thu, 10 Mar 2016 12:25:58 +0200 >>> Jani Nikula <jani.nikula@xxxxxxxxx> escreveu: >>> >>>> TL;DR? Skip to the last paragraph. >>>> >>>> On Wed, 09 Mar 2016, Mauro Carvalho Chehab <mchehab@xxxxxxxxxxxxxxx> wrote: >>>>> I guess the conversion to asciidoc format is now in good shape, >>>>> at least to demonstrate that it is possible to use this format for the >>>>> media docbook. Still, there are lots of broken references. >>>> >>>> Getting references right with asciidoc is a big problem in the >>>> kernel-doc side. As I wrote before, the proofs of concept only worked >>>> because everything was processed as one big file (via includes). The >>>> Asciidoctor inter-document references won't help, because we won't know >>>> the target document name while processing kernel-doc. >>> >>> I was able to produce chunked htmls here with: >>> >>> asciidoctor -b docbook45 media_api.adoc >>> xmlto -o html-dir html media_api.xml >>> >>> The results are at: >>> https://mchehab.fedorapeople.org/media-kabi-docs-test/asciidoc_tests/chunked/ >>> >>> But yeah, all references seem to be broken there. It could be due to some >>> conversion issue (I didn't actually tried to check what's wrong there), >>> but I think that there's something not ok with docbook45 >>> output for multi-part documents (on both AsciiDoc and Asciidoctor). >>> >>>> Sphinx is massively better at handling cross references for >>>> kernel-doc. We can use domains (C language) and roles (e.g. functions, >>>> types, etc.) for the references, which provide kind of >>>> namespaces. Sphinx warns for referencing non-existing targets, but >>>> doesn't generate broken links in the result like Asciidoctor does. >>>> >>>> For example, in the documentation for a function that has struct foo as >>>> parameter or return type, a cross reference to struct foo is added >>>> automagically, but only if documentation for struct foo actually >>>> exists. In Asciidoctor, we would have to blindly generate the references >>>> ourselves, and try to resolve broken links ourselves by somehow >>>> post-processing the result. >>>> >>>>> Yet, from my side, if we're willing to get rid of DocBook, then >>>>> Asciidoctor seems to be the *only* alternative so far to parse the >>>>> complex media documents. >>>> >>>> I think you mean, "get rid of DocBook as source format", not altogether? >>>> I'm yet to be convinved we could rely on Asciidoctor's native formats. >>> >>> What I mean is that, right now, I see only two alternatives for the >>> media uAPI documentation: >>> 1) keep using DocBook; >>> 2) AsciiDoc/Asciidoctor. >>> >>> Sphinx doesn't have what's needed to support the complexity of the >>> media books, specially since cell span seems to be possible only >>> by using asciiArt formats. Writing a big table using asciiArt is >>> something that is a *real pain*. Also, as tested, if the table is >>> too big, it fails to parse such asciiArt tables. So, while Sphinx >>> doesn't have a decent way to describe tables, we can't use it. >> >> >> Huge tables and cell-spans are the *real pain* ;-) ... with sphinx-doc, >> (mostly) you have more then one choice .. e.g. import csv tables .. >> but this should be discussed by example ... >> >> >>> If it starts implementing it, then we can check if the other >>> features used by the media documentation are also supported. >>> Probably, multi-part books would be another pain with Sphinx. >>> We have actually 4 books inside a common body. A few chapters >>> (like book licensing, bibliography, error codes) are shared >>> by all 4 documents. >>> >>> But, so far, I can't see any way to port media books without >>> lots of lot of work to develop new features at the Sphinx code. >> >> >> may I can help you ... >> >> >>>> The toolchain gets faster, easier to debug and simplified a lot with >>>> DocBook out of the equation completely. Sphinx itself is stable, widely >>>> available, and well documented. IMO there's sufficient native output >>>> format support. There are plenty of really nice extensions >>>> available. There's a possibility of doing kernel-doc as an extension in >>>> the future (either by calling current kernel-doc from the extension or >>>> by rewriting it). >>> >>> Well, if we go to Sphinx for kernel-doc, that means that we'll need >>> 2 different tools for the documentation: >>> - Sphinx for kernel-doc >>> - either DocBook or Asciidoctor/AsciiDoc for media. >>> >>> IMHO, this is the worse scenario, as we'll keep depending on >>> DocBook plus requiring Sphinx, but it is up to Jon to decide. >>> >> >> The migration of kernel-doc is a long term project, not a >> one shot job. The scope of documents to migrate is not limited >> to the files with DocBook markup in, most documents have not >> a real markup. >> >> Please take a look at my thoughts and efforts about migration. >> >> * https://sphkerneldoc.readthedocs.org >> >> * https://github.com/return42/sphkerneldoc.git >> >> sphkerneldoc.git is a small project started this weekend, within >> this project I show you, how migration could be done and >> we can discuss concerns like "tables and cell-spans" by example. >> >> Believe me, most concerns discussed in this thread are a leak of >> knowledge. I'am working with sphinx-doc since 7 years, switched >> over from DocBook (escaped from a 8 years lasting XML hell). >> DocBook and sphinx-doc are complete different, so sphinx-doc >> might feels odd in the first time, but if you have switched >> like me, you will never go back again. >> >>>> Dan keeps bringing up the active community in Asciidoctor, and how >>>> they're fixing things up as we speak... which is great, but Sphinx is >>>> here now, packaged and shipping in distros ready to use. It seems that >>>> of the two, an Asciidoctor based toolchain is currently more in need of >>>> hacking and extending to meet our needs. Which brings us to the >>>> implementation language, Python vs. Ruby. >>>> >>>> I won't make the mistake of comparing the relative merits of the >>>> languages, but I'll boldly claim the set of kernel developers who know >>>> Python is likely larger than the set of kernel developers who know Ruby >>>> [citation needed]. AFAICT there are no Ruby tools in the kernel tree, >>>> but there is a bunch of Python. My own very limited and subjective >>>> experience with other tools around the kernel is that Python is much >>>> more popular than Ruby. So my claim here is that we're in a better >>>> position to hack on Sphinx extensions ourselves than Asciidoctor. >>> >>> Sorry, but I don't buy it. Python is, IMHO, a mess: each new version >>> is incompatible with the previous one, and requires the source to >>> change, in order to use a newer version than the one used to write >>> the code. So, when talking about Python, we're actually talking about >>> several different dialects that don't talk well to each other. >> >> Sorry, you are complete wrong ... I'am 15 years python programmer, >> shipped out huge projects with my customers ... we never have seen >> these problems ... sorry ... >> >> >>> I don't know about Ruby. So far, I don't have anything against (or in >>> favor) of it. I bet most Kernel developers would actually prefer a >>> toolchain in C. If such tool doesn't exist, anything else seems >>> equally the same ;) >> >> Why we are talking about script languages? What needed is a >> authoring system, which is as near as possible to the developers, >> which are the authors. >> >> Sphinx-Doc is a standard authoring-tool versioned, maintained >> and extended by thousands of developers ... >> >> >>>> My conclusion is that Sphinx covers the vast majority of the needs of >>>> our documentation producers and consumers, in an amazing way, out of the >>>> box, better than Asciidoctor. >>>> >>>> Which brings us to the minority and the parts where Sphinx falls short, >>>> media documentation in particular. It's complex documentation, with very >>>> specific requirements on the output, especially that many things remain >>>> exactly as they are now. It also feels like the target is more to have >>>> standalone media documentation, and not so much to be aligned with and >>>> be part of the rest of the kernel documentation. >>>> >>>> I want to question the need to have all kernel documentation use tools >>>> that meet the strict requirements of the outlier, when there's a better >>>> alternative for the vast majority of the documentation. Especially when >>>> Asciidoctor isn't a ready solution for media documentation either. >>>> >>>> In summary, my proposal is to go with Sphinx, leave media docs as >>>> DocBook for now, and see if and how they can be converted to >>>> Sphinx/reStructuredText later on when we have everything else in >>>> place. It's not the perfect outcome, but IMHO it's the best overall >>>> choice. >>> >>> Well, this could be done. We don't have any good reason to move >>> the media docs out of DocBook. >> >> Sorry but again wrong: you lost many of the authors which are >> frustrated by a XML markup and you lost many developers to improve >> the toolchain, frustrated by a complicated DocBook-XML XSLT >> toolchain with SGML markup from the middle of the last epoch. >> >>> On the contrary, this means an extra >>> work. The only advantage is that it is a way simpler to write >>> documentation with a markup language, but converting from the PoC >>> to its integration at the Kernel tree still require lots of work, >>> specially due to the cross-refs "magic" scripts that we have under >>> Documentation/DocBook/media/Makefile. >> >> Yes, you are right, migration is a process not a one shot >> job, as I mentioned before. You are a great programmer, your >> documentation is also great, this invest should be preserved. >> So lets take a try. It would be a honor for me to show >> you all theses steps by example on my repository (see above). >> >>> As I said, the only big drawback is to keep depending on two >>> different tools for kernel-doc and for media documentation. >> >> -- Markus -- >> >> >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-media" in >> the body of a message to majordomo@xxxxxxxxxxxxxxx >> More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
