Sure, similar problem would affect almost any html generators, API documentation included. Rubygem static data are much smaller than rtd theme content. I did not know about web assets guidelines [1], but they look reusable. They already contain fonts in common path. Your example has advantage, because it uses only ttf fonts. sphinx_rtd_theme uses only woff format, which is prohibited by assets [2]. Interesting enough, it uses _only_ woff2 and woff, does not try ttf at all in rtd theme. rtd theme fonts are much bigger, only fonts has 3,2 MB. 16K /usr/share/doc/python-sphinx_rtd_theme-doc/html/_static/js 3,2M /usr/share/doc/python-sphinx_rtd_theme-doc/html/_static/css/fonts 3,3M /usr/share/doc/python-sphinx_rtd_theme-doc/html/_static/css 3,9M /usr/share/doc/python-sphinx_rtd_theme-doc/html/_static/ -rw-r--r--. 1 root root 281K 19. led 2021 jquery-3.5.1.js -rw-r--r--. 1 root root 88K 19. led 2021 jquery.js -rw-r--r--. 1 root root 35K 19. led 2021 underscore-1.3.1.js -rw-r--r--. 1 root root 16K 19. led 2021 searchtools.js -rw-r--r--. 1 root root 16K 4. led 2021 logo-wordmark-light.svg -rw-r--r--. 1 root root 14K 28. led 2021 basic.css -rw-r--r--. 1 root root 12K 19. led 2021 underscore.js -rw-r--r--. 1 root root 11K 28. led 2021 language_data.js -rw-r--r--. 1 root root 9,2K 19. led 2021 doctools.js If every doc package contains unnecessary 4MB, how could we reduce if all reused common content? #dnf repoquery --whatprovides '*/_static/underscore.js' | wc -l 613 613*3916/1024/1024 =~ 2.29 GB of wasted space, counting only single build of each package. That seems not unimportant, if we ignore legal problem at all. I know, disk space is not so expensive, but this does not seem tiny. 1. https://docs.fedoraproject.org/en-US/packaging-guidelines/Web_Assets/ 2. https://docs.fedoraproject.org/en-US/packaging-guidelines/Web_Assets/#_fonts 3. https://github.com/readthedocs/sphinx_rtd_theme/blob/master/src/sass/_theme_font_local.sass On 2/8/22 20:00, Vít Ondruch wrote: > Just FTR, the same issue suffers rubygem-*-doc subpackages. While this > is long time unresolved issue, I am thinking about providing Fedora > specific template for the documentation, which would be available as > independent package in Fedora (properly refering to fonts and js > libraries). This would allow the generated documentation to safely > symlink everything. > > While I don't know anything about Shinx, I'd like to believe that > similar approach would be feasible. > > Vít Sure, I visited rtd theme upstream [3]. It seems it only references woff font format, which should not be used according to guidelines. That means it either would need downstream change or separate fedora shared package, which would use better paths and ttf format. I do not yet understand process of variable expansions of sphinx_rtd_theme npm processing and then how it selects source font paths and matching destination. I thought creation of common package with symlinks to shared content would be enough. Then preprocessed by some script, which would replace copies with links to shared space. But modification of minimized css styles seems to be wrong. I guess different template would be cleaner approach. Regards, Petr > > > Dne 08. 02. 22 v 19:50 Petr Menšík napsal(a): >> I prefer much more HTML documentation than PDF. While I try to make PDF >> also available, HTML is more useful in offline situations, on a train >> for example. >> >> On 2/8/22 15:18, Daniel P. Berrangé wrote: >>> On Mon, Feb 07, 2022 at 04:51:35PM +0100, Petr Menšík wrote: >>>> Hi! >>>> >>>> I maintain bind package, which generates html documentation using >>>> sphinx >>>> and sphinx_rtd_theme. I admit this format is quite popular. Once I >>>> have >>>> noticed that bind-doc package is quite big. When looking why, I have >>>> found not a small number of static copies were generated by >>>> documentation process. >>>> >>>> If I remember correctly, fonts are allowed to be shipped only by font >>>> packages. Not only sphinx packages ships static copies of javascript >>>> underscore and jquery, but it seems every such package includes >>>> also set >>>> of fonts contained in its documentation. >>> Perhaps I'm mis-interpreting the guidelines, but I don't feel >>> like the font packaging rules should apply in this case. When >>> we describe the rules for packaging fonts, I feel like that was >>> related to fonts that are to be installed for shared usage. >>> ie stuff going into /usr/share/fonts should be provided by a >>> fonts-xxx package, not as a side effect of some other package. >> I admit I thought fonts are allowed to be packaged ONLY in fonts package >> and should be reused otherwise. That may not be required. >>> The fonts that get copied/around embedded in documentation are >>> for private usage by only that documentation. Any shared docs >>> generator tools would ideally pull their fonts from common source, >>> but whatever they do for output we should just accept. It is not >>> a sane use of resources to expect package maintainers to hack the >>> generated docs to change how they deal with fonts. Sphinx is just >>> one docs tool, there are many other tools, often written just to >>> suit the one app in question. >>> >>> Regards, >>> Daniel >> Thanks to Ben Beasley for pointing relevant thread in packaging list, it >> revealed the problem is not primary about size wasted by repeated copies >> of static content. Especially fonts are included without any hint of >> original license. It is non-trivial to obtain original data license and >> I am certain most packages ignores it instead of following packaging >> guidelines precisely. >> >> I found there are python-sphinx-doc and python-sphinx_rtd_theme-doc, >> which are generated by their respective generators. >> >> I compared their contents: >> >> diff -ru /usr/share/doc/python-sphinx-doc/html/_static/ >> /usr/share/doc/python-sphinx_rtd_theme-doc/html/_static/ >> >> That revealed most of files are not modified template, but exact copy. >> Just documentation_options.js and pygments.css seems to be generated and >> modified. >> >> Could we perhaps create python-sphinx-common package, which would put >> common data to /usr/share/doc/python-sphinx-common/_static/ for example. >> Then we could make a script for packagers to convert bundled copies to >> links. sphinx-package make-shared <path-to-generated-doc-dir> >> >> Such script would remove copies and create relative symlinks to >> /usr/share/doc/python-sphinx-common/_static/ content directory. Packager >> would then just ensure his doc package Requires: python-sphinx-common >> package. Harder work would be done by sphinx packager, which would >> ensure common package has all needed Requires: to fonts or js-jquery, >> js-underline or similar. He or she would make sure those files stay in >> single path and point to minimized or normal from original js library. >> Also would ensure they do not depend on python version for example. >> >> Strange thing is, jquery bundled in sphinx package is more recent than >> matching js-jquery version in f35. Difference is minimal, but they are >> not the same. Opposite situation is with js-underline, which is much >> higher version in js-underline than the one bundled in sphinx. Even >> original packages do not Provides: bundled(js-jquery) from built doc >> package. >> >> If self-contained copy were required, it might not be so hard to prepare >> also sphinx-package make-static action, which would dereference symlinks >> and replace them with hard link if possible, a file copy otherwise. >> >> sphinx_rtd_theme-package might specialize and extend copied list of >> files. It would have its own common subpackage with correct >> dependencies. I think such system would allow proper licensing, because >> it would point to original package with its well specified license. It >> would not work great in case of subdirectories mounted somewhere else. >> Script converting shared links to static copies should make it >> reasonably simple to create a workaround. >> >> I think that might solve both repeated bundling of JS libraries and >> fonts and save visible amount of space. I admit I am not maintainer of >> sphinx nor rtd_theme, so most of work would be done outside of my >> packages. Is FESCO okay with bundled javascript libraries in similar >> packages? >> >> Regards, >> Petr >> > > _______________________________________________ > packaging mailing list -- packaging@xxxxxxxxxxxxxxxxxxxxxxx > To unsubscribe send an email to packaging-leave@xxxxxxxxxxxxxxxxxxxxxxx > Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: https://lists.fedoraproject.org/archives/list/packaging@xxxxxxxxxxxxxxxxxxxxxxx > Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure -- Petr Menšík, Fedora project contributor PGP: DFCF908DB7C87E8E529925BC4931CA5B6C9FC5CB _______________________________________________ packaging mailing list -- packaging@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to packaging-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/packaging@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
