On 1/20/25 4:20 AM, Daniel P. Berrangé wrote:
On Mon, Jan 20, 2025 at 11:29:55AM +0100, Michael J Gruber wrote:Mikel Olasagasti venit, vidit, dixit 2025-01-20 08:02:18:Hi, Go-SIG has raised a ticket with FESCo [1] to propose a significant shift in Fedora's packaging approach for Go dependencies: moving to vendoring/bundling by default. This would represent a major departure from our current guidelines [2]. Given the potential impact, it was suggested that we bring this topic to the devel-list for broader input and an open discussion. The rationale for this proposal stems from the increasing challenges in maintaining Go dependencies under the current model, including issues with timely reviews, dependency management, and the impact of some "core" orphaned packages. Allowing vendoring by default could help alleviate these bottlenecks and improve the stability of Go-based packages in Fedora.I understand that these are challenges, and they are increasing because there are so many upcoming ecosystems which build their own distribution channels and tools, be it Go, Rust, Julia, even Python to some degree (think pip, conda and the like). And indeed, as Jan's reply shows, once we allow bundling in general for one language people will want it for other language ecosystems, and "rightly" so, at least as long as the same reasoning applies.Yes, I think we should apply the logic consistently across languages. They might not have all reached the same point of frustration, but the challenges are clearly common across many of these languages which are dealing with non-ELF packages with upstream bundling or locked dep versions as the default.Following the Go-SIG proposal would be the can-opener to deviate from our distribution model in general. I mean, if I can pull in a myriad of Go or Rust modules for my package to build (without individual review or re-review in case of changed dependencies), then why should I bother unbundling a library such as zlib, regex, openssl, ... ?IMHO comparing/equating these non-C/non-ELF language modules, to traditional ELF libraries was/is a mistake. The biggest difference is simply one of granularity. A non-trivial ELF library will provide 1000's of APIs across a vast range of functionality in a single library. In these non-C languages, modules are often incredibly small just providing a handful of APIs. IOW, functionality that was a single ELF library, may be spread across 100's of python/go/rust modules. This granularity then feeds the other problems and/or design approaches. Many deps are fast moving with a somewhat more relaxed approach to API compat than traditional ELF libraries. With 100's of deps, fixing on specific versions quickly becomes requirement to get a stable foundation for the app which can be tested without deps changes breaking stuff too often. Fedora went down the route of applying our existing unbundling practices from ELF libraries across these non-C ecosystems for various good reasons, but this level of module granularity & differing attitudes to the module API lifecycle is drowning the distro maintainers in work. At the same time it risks delivering something which is of worse quality than that from upstream, because our versions won't match what upstream has validated in their CI systems. IMHO it is right to ask whether we have the right trade-off here. We rely on the heroics of small numbers of SIG members to keep this all working, and when key members of the SIGs change their focus it is at risk of falling apart. See NodeJS, or Java ecosystem changes in Fedora. This isn't a sustainable model. Any 3rd parties who want to build on top of what Fedora ships have a foundation that could collapse with any new major Fedora release. It is no surprise that application developers have embraced containers as a delivery mechanism in a way that they rarely embraced distros directly in the past. If we spent less time unbundling all these non-ELF modules, our maintainers would be freed up to work on other things that could potentially be more beneficial Fedora users. That isn't to say Fedora's process is without value. There are key parts of our review process that are critically beneficial. Most notabley license scanning has had clear benefits both to Fedora and the upstreams, with many problems uncovered & fixed over the years. Fedora & upstream aligned in their desirables in this case (mostly), so it has been a success. With unbundling by contrast, Fedora & upstreams are essentially completely non-aligned, and I can't see that changing - all signs are that we are getting further apart for non-infrastructure components of the OS in the non-C/ELF world. The biggest challenge with bundling is the constraint it imposes on our ability to deliver security fixes. Identifying the existance flawed code can be automated to a reasonable degree since we have metadata to track what bundled versions exist. It is the actual delivery of fixes that is hard, for the very same reasons why unbundling is hard to being with. The fixed deps are often not amenable to easy replacement. Overall I support allowing bundling of all non-ELF packages, but we shouldn't do that in isolation. How can Fedora adapt to this reality, to maintain the aspects of Fedora that deliver good user value, while relaxing/eliminating the aspects that create busy-work. Our package centric approach to reviewing might benefit from adaptations. eg should bundled packages get their own distinct mini-reviews that we track as formal parts of the process. These mini-reviews could be reused for future pacakges with common bundled pieces. We would also need to consider what our committement is to security updates when apps are bundling stuff, as the same reasons that make unbundling huard, also make fixing security issues hard. If we are not doing busy-work on unbundling though, perhaps we'd have more time to invest on security updates & figuring our automation to help when fixing bundled deps. With regards, Daniel
Attachment:
OpenPGP_signature.asc
Description: OpenPGP digital signature
-- _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
