One last follow-up:
It turns out that even the situation with Doxygen is worse. The file
jquery.js inserted by Doxygen was not just a copy of JQuery (js-jquery)
to be unbundled; it also contained in the same file:
- JQuery UI (js-jquery-ui)
- jquery.ScrollTo (https://github.com/flesler/jquery.scrollTo/)
- PowerTip (https://stevenbenner.github.io/jquery-powertip/)
- JQuery UI Touch Punch (js-jquery-ui-touch-punch)
- SmartMenus jQuery Plugin (http://www.smartmenus.org)
It’s possible to account for all of these, package the missing ones
separately, and reconstruct the bundle using unminified sources (or
patch the generated HTML to use separate script tags, yuck), but the
burden is getting ridiculous again. It looks like dropping all
Doxygen-based HTML documentation, or building a less-useful PDF instead,
will be the sensible (but disappointing) outcome.
On 9/26/21 10:57, Ben Beasley wrote:
I’ve used cairomm as a case study for trying to bring a Doxyen-based
HTML documentation package into compliance, and python-autodocsumm as a
case study for Sphinx.
-----
For cairomm, I have unbundled js-jquery and altered the License for the
-doc subpackage. I am not treating the Doxygen-generated JavaScript
files as bundled libraries because they are templated and
project-dependent, not simply copied in as fully-reusable libraries.
While Doxygen does not insert a standalone license file for its MIT
license, the text is at least present in the JavaScript file headers. I
might have missed mentioning some non-JavaScript files inserted by
Doxygen that are potentially MIT-licensed but do not reference their
license.
Another solution would be to ask Doxygen to build a PDF instead of HTML.
The result would have no visible bundled assets, but in my opinion would
be less convenient for users.
There are some judgement calls here, but I think the result
(https://src.fedoraproject.org/rpms/cairomm/c/948e04f946d95612948d81d9090e2bcfa95abe56?branch=rawhide)
is reasonable.
-----
The situation for python-autodocsumm’s Sphinx documentation is more
dire. I can unbundle the Sphinx-inserted copies of js-jquery and
js-underscore. I can add virtual Provides for Sphinx’s libraries
doctools.js and searchtools.js:
Provides: bundled(js-sphinx-doctools)
Provides: bundled(js-sphinx-searchtools)
although I cannot version them, because the version could only be that
of Sphinx itself, and that changes regularly without alterations to my
package’s sources; I don’t know a way to handle that automatically, and
I could not reasonably keep it synchronized manually. These libraries
were never intended to be handled separately from the documentation they
support.
I can claim that language_data.js and documentation_options.js are not
bundled libraries, because they are generated from templates using
project-dependent options.
Now I have the problem that all of the Sphinx-inserted files in _static
have the same BSD license as Sphinx itself, but they only reference the
Sphinx LICENSE file in their headers; they do not contain license text
at all, and BSD is one of the licenses where license text is required.
I could possibly handle this by copying in the license file from the
Sphinx package in the build environment. I’d better make sure I don’t
miss a hypothetical Sphinx license change, so I should probably grep
that license file in %check for a key phrase to make sure its contents
are what I expect.
Now I’m left with the theme. The autodocsumm documentation uses
sphinx_rtd_theme, which is a very common choice. Theme files are mostly
in _static/css/ and _static/js/.
In _static/js/, there are two JavaScript files, badge_only.js and
theme.js. Both are minified and contain no license information. Okay, I
can check the license of python3-sphinx_rtd_theme, and maybe I can copy
in the unminified versions manually to comply with that part of the
guidelines. Oh, it turns out only the minified versions are packaged, so
there’s nothing I can do. (Well—shipping only minified files isn’t
compliant with guidelines, so I could send a PR or file an issue on
python-sphinx_rtd_theme and expect the maintainer to do something about
it. I’m not trying to call out this particular package or its
maintainers, as it’s very hard to find and deal with all the bits of
JavaScript and web content that are tucked away in Python packages these
days, and such oversights are rampant in existing packages.)
In _static/css/, there are two corresponding CSS files, badge_only.css
and theme.css. Both have the same problems (minified only, no license
information), but *also*, theme.css has a bundled copy of the CSS from
Font Awesome 4.7.0. That’s also MIT-licensed, with only a URL for the
license and no included license text.
-----
This is the point at which I say that the burden of trying to package
this near-minimal example of Sphinx-generated documentation in strict
compliance with the guidelines is unreasonable, and I drop it, along
with the documentation in all other Python packages for which I am
responsible.
I would be surprised if there is even one case where HTML documentation
for a Python package is currently packaged in a way that truly satisfies
all of these requirements around license text, web assets, JavaScript,
and bundling; and I suspect the broader situation for Doxygen and other
documentation generators is not much better.
As a user who has often needed to work offline for various reasons, it
would be a great loss if the solution really has to be that HTML
documentation should not be packaged in general.
On 9/24/21 09:21, Ben Beasley wrote:
It was recently brought up in a package review
(https://bugzilla.redhat.com/show_bug.cgi?id=2006555) that a
Sphinx-generated HTML documentation package contained several
JavaScript libraries, and that:
1. their licenses should be accounted for and added to the License
field for the -doc subpackage
2. they should be treated as bundled libraries and marked with virtual
Provides
Note that the JavaScript is inserted by the Sphinx documentation build
system, and is not present in the source tarball. Your “typical”
Sphinx documentation has minified and unminified copies of JQuery
(js-jquery) and Underscore (js-underscore), both MIT-licensed, plus
several unminified utility libraries that belong to Sphinx itself
(doctools.js, language_data.js, searchtools.js). The overall Sphinx
project is BSD-licensed. The details vary by theme, but this is the
most common case.
To me, it seems that this feedback is a reasonable strict reading of
the relevant packaging guidelines. If this is the right interpretation
of Fedora policy, what should be done about it in general?
-----
Based on looking for installed files named “searchtools.js” in
packages named -doc or -docs, there are about 600 documentation
packages generated with Sphinx. This might miss some that have exotic
themes.
As far as I can tell, only one existing package in the distribution,
python-murano-package-check, tries to use virtual Provides to track
the libraries that belong to Sphinx itself:
Provides: bundled(js-doctools)
Provides: bundled(js-jquery) = 3.1.0
Provides: bundled(js-searchtools)
Provides: bundled(js-underscore) = 1.3.1
Provides: bundled(js-websupport)
and this package does not actually build its documentation subpackage.
Only the following packages have virtual Provides for js-jquery and
js-underscore. In most cases these lack the version numbers.
- arb-doc
- mpdecimal-doc
- python-BTrees-doc
- python-latexcodec-doc
- python-networkx-doc
- python3-persistent-doc
- sympy-doc
Of the packages in the list above, only mpdecimal-doc has a License
field that seems to try to account for the licenses of the JavaScript
libraries.
-----
Personally, I can add virtual Provides and adjust the License on the
-doc subpackage for my package under review, and it’s even possible
for me to replace the Underscore and JQuery libraries with symbolic
links to those installed by js-jquery and js-underscore. However,
changing one package, or even every package I control, is a tiny drop
in a vast ocean of documentation packages. Consider, for example, that
Doxygen also includes JQuery and several Doxygen-specific libraries in
its output, so the same arguments about licenses and virtual Provides
apply to it. A search for “dynsections.js” in packages named *-doc or
*-docs gives nearly 400 Doxygen-based packages.
If this is a real problem, it seems like it needs to be handled and/or
documented in a way that can potentially scale to the rest of those
600 Sphinx-based packages, to the nearly 400 Doxygen-based packages,
and beyond that to the output of other documentation systems. I am not
sure what that would look like.
All thoughts are welcome.
– Ben Beasley (FAS: music)
_______________________________________________
packaging mailing list -- packaging@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to packaging-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/packaging@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
_______________________________________________
packaging mailing list -- packaging@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to packaging-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/packaging@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
