On 13.2.2015 02:11, Zbigniew Jędrzejewski-Szmek wrote: > On Thu, Feb 12, 2015 at 01:32:04PM -0500, Stephen Gallagher wrote: >> (Logistical note: please keep all replies to this thread on >> devel@xxxxxxxxxxxxxxxxxxxxxxx) >> >> tl;dr Shall we consider requiring a lesser package review for packages >> that are not present on Product or Spin install media? > Despite the title, and the tl;dr summary, what you are proposing boils > down to a blanket exception on bundling. This is both not enough and > dangerous. I agree that we could relax the rules in some places, where > the effort of conforming to Guidelines is just too big for the real > gain. But I think the way this is done should be much more subtle, > based on an analysis of individual tradeoffs. > > In your proposal the ring of the package is the important thing. But > the dependency tree is a very well connected graph, and various fringe > packages are used by "core" packages, so there's no way to develop > anything using just the core, and any real-life server is also going > to include packages from all rings. Security bugs in "fringe" > web-facing services are more important than bugs in "core" tools > which are purely local. For example, recent XSS vulnerability in > jquery [1] nicely cuts through all rings. Unbundling jquery here would > give a huge benefit, and does not become less important the farther a > package is from the core. > > [1] https://bugzilla.redhat.com/show_bug.cgi?id=1166041 > >> == Premise == >> >> So, some time ago, we started talking about dividing up the Fedora >> package set into a series of "rings". One of the driving purposes behind >> this idea was to re-evaluate our policies for packaging in each of these >> rings. A fair amount of time has passed and no formal discussions have >> taken place (that I have seen, at any rate), so I'm going to provide >> here a simple "starter" proposal that we can work from and see where it >> leads. >> >> To begin, I'll try to identify some of the major advantages and >> disadvantages in our current policies. (In no particular order...) >> >> === Advantages === >> * Consistency: every shared library, language-specific module and >> application should have packaging consistent with others in its >> category. This makes it easier for comaintainers and provenpackagers to >> pick it up and work on it. >> >> * The no-bundled-libraries policy means that when a library module >> requires an update, it means that only one package needs to be modified >> in order to enhance all applications on the system that consumes it. >> This is a significant time-saver when it comes to dealing with >> (increasingly common) security vulnerabilities. >> >> * Package review "front-loads" the task of educating the packagers. It >> guarantees that packages enter the collection by meeting a very high >> standard. This does a great deal to accomplish the "consistency" >> mentioned above. >> >> * Legal review and the FPCA ensures that Fedora remains true to its >> "Freedom" Foundation (as well as protecting Fedora contributors from the >> perils of the international legal system). >> >> === Disadvantages === >> * Very strict policies often leads to potential packagers giving up. > True. But packagers often give up for many other reasons, including > unresolved legal problems with the software, many different packaging > problems other than bundling, lack of sponsors, etc. > >> * Package reviews for less-interestingackages (such as those for less >> popular SIGs) often remain un-reviewed for weeks, months or even years. >> >> * The package policies are only ever reviewed during the initial >> creation of the package. Once that initial (high) hurdle is cleared, the >> packager essentially has free reign to do whatever they want with their >> package. This sometimes means that as time passes, the spec files >> "bit-rot" or otherwise start accumulating other inconsistencies. (A >> common example is the package whose upstream starts bundling a library >> without the packager noticing). > Your proposal would only worsen, not fix that. > >> * Many upstream projects do not concern themselves with being "in" any >> particular distribution (with the notable example being the >> Debian/Ubuntu flavors which have amassed a sufficient apparent userbase >> that they sometimes get special treatment). For a variety of reasons, >> this often leads directly to bundling the vast majority of their >> dependencies. This is done for many reasons, but the two most common are >> supportability and portability; it's impossible for many upstreams to >> actually QA their package with every possible distro library. Instead, >> if they ship everything they depend on, they can guarantee *that* >> specific combination. This moves the responsibility from the >> distribution to the upstream package to maintain their bundled >> libraries. > I think this responsibility is an illusion. Most upstreams only care > that the bundled versions allow their package to continue to work, and > are not at all capable of fixing vulnerabilities or even other bugs. > > I think that it is also very unlikely that packages would be properly > reviewed when they move to an inner ring. Look at how well the merge > reviews are going. > >> * With many languages, there is no possibility of installing multiple >> versions of the same library on the same system, so if an application >> requires a newer or older version of the library than is in Fedora to >> run, it fails. For those languages that *do* allow this, it still adds a >> significant maintenance burden on the packagers to keep multiple >> versions of the same package up-to-date (which gets particularly hard >> when upstream abandons a version that something else in Fedora depends >> on...) > In practice, quite often it turns out that the version dependency > is not as strict as upstream makes it out to be. Recent example: > ElasticSearch [2]: despite initial concerns, it seems that ElasticSearch > will not require custom versions of any packages. Another > example: dateutil update [3]. Out of the many packages declaring > a dependency on dateutil 1.5, about 50% have been checked, and > they all work just fine with dateutil 2.4. > > [2] https://bugzilla.redhat.com/show_bug.cgi?id=902086 > [3] https://bugzilla.redhat.com/show_bug.cgi?id=1126521 > >> == Proposal == >> With these things in mind, I'd like to propose that we amend the >> packaging policy by splitting it into two forms: >> >> === Core Packages === >> Any package that is provided on a release-blocking medium (which at >> present includes Fedora Atomic, Fedora Cloud, Fedora Server, Fedora >> Workstation, the KDE Spin and several ARM images) must comply exactly >> with the packaging guidelines as they are written today. These packages >> must receive a full package review *when they are added to the install >> media*. Any package present on the media if this proposal is adopted is >> exempted from this requirement. Any package to newly appear on the >> install media after this time *should* (I hate that word...) receive a >> new package review, even if it is already present in Fedora. >> >> === Ring Packages === >> Any new package that is *not* going to be part of the install media set >> is required to pass a lighter review and is permitted to carry bundled >> libraries, with caveats to be listed below. >> >> ==== Ring Package Review Requirements ==== >> * A reviewer *MUST* establish suitability for the Fedora Project. This >> means verifying that the package contains only permissible content >> (legally-speaking). >> >> * The package *MUST* conform to the FHS and other filesystem conventions >> used by Fedora (such as %{python3_sitelib}), except when granted an >> exception by the FPC. >> >> * The package *MAY* contain bundled libraries or other projects, but if >> it does so, it *MUST* contain a "Provides: bundled(pkg) = version" for >> each such bundling. This is done so that we can use the meta-data to >> identify which packages may be vulnerable in the event of a security >> issue. > I'd like to make a counterproposal: analyze what Packaging Guidelines > cause the most problems, and periodically check if they are still worth > the effort. In specific cases, allow the package to be approved, but > open a bug against the package to keep the issue open and hopefully fix > it in the future. > > Requirements which I'd drop right now: > > 1. strict adherence to FHS. There's only one /usr now, so the split > between /usr/share, /usr/lib, /usr/libexec is obsolete. > > 2. forbidding JNI libs in %{_libdir}. This is very popular, requires > modifying the source of the package to fix, and provides little > benefit. Instead it would be enough to require the JNI .so name to > not conflict. For example suffixing with 'j' would be enough. > > I'm sure there are other things like that, which are pain in the ass > to fix, and could be dropped without loss to the distribution, in > contrast to a blanket exception to the guidelines for a large subset > of packages. > > Zbyszek I very much agree with Zbyszek's word above. If the intent is to make Fedora attractive it makes sense to relax packaging guidelines at points where the change gives only small disadvantage to distro but makes life of packagers easier (see the JNI example). If others feel that anti-bundling policy is too hard for new packages I would prefer to relax it step by step and *do not* start with blanked bundling exception. Modified version of Zbyszek's idea with time constraints follows: 1) Accept the new package into Fedora N even with bundled libraries. 2) Open bug for every bundled library. 3) Set bug deadline to Fedora N+1. 4) At some point in Fedora N+1 release preparation process: Check if all bundling exception bugs opened in Fedora < N are closed. If not, remove affected components which did not comply from ditro. This will result in some fancy policy like 'no cross-SRPM dependencies on components with bundled libraries are allowed or something similar'. Maybe these packages with temporary bundling exception can be in separate 'fedora-pkg-candidate' repo? This basically limits number of packages with bundled libraries in one release. It is an attempt to balance: - Headache caused by security issue in a library. - Time needed to get new packages to Fedora. - Incentive for unbundling. Personally I consider point "Time needed to get new packages to Fedora." to have major importance. The fact that the package is in repo/release gives you (or at least me :-) great motivation to spend more time on it and possibly work on library unbundling. The time-bounded nature of the exception protects distro from becoming the same mess as Windows with indefinite DLL bundling. -- Petr Spacek @ Red Hat -- devel mailing list devel@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
