On Wed, Nov 30, 2022 at 11:54:10AM +0000, Peter Robinson wrote: > Hi Fabio, > > Been meaning to reply to this, but it got lost in the mail pile. > > > > But running `cargo fetch` with a clean cache pulls down *390* crates. Of > > > these, it looks like 199 (!) are already packaged as rust-[crate]-devel, > > > which is *amazing*. But... that still is hundreds that I'd have to add. And > > > mostly they are things I don't know _anything_ about. > > > > You must realize that this is an extreme case. For many Rust > > applications that people want to package for Fedora, the number of > > dependencies that are missing is rather small, *because* most popular > > libraries are already packaged. > > In my experience, and I'm not packaging any form of gaming engine, > it's not an extreme case, for a few small things and one slightly > larger thing I *have* packaged I now maintain over 60 more packages. I > have another one I want to package and AFAICT I get to package another > 50-100 but frankly it's hard to tell, and this is something I have > control over and have been actively trying to do upstream reduction of > deps. The magnitude of the number of deps is the real killer for not only Rust, but also Go, and arguably anything that isn't a C based application. Packaging up all of a project's deps individually is viable when there are a relatively small number of them, especially if most of the deps are shared by other apps, and the libraries have good API + ABI stability promises. This is common in the case of most C applications/libraries, since shared libraries are relatively speaking fairly broad in functional scope, and often long lived, because that is the mindset of the C ecosystem in general. Modern languages & their ecosystems though have brought about a world where the granularity of deps is waaaaay smaller, where the focus is on being able to evolve rapidly, with less focus on long term stable API/ABI, as apps are expected to just fix against specific versions they've tested on. So functionality that lives in single library in C, often ends up being spread across 20-200 modules in any of the modern non-C languages. We've been experiancing this problem in Fedora for a very long time and while we have spec auto-generators, that still leaves masses of busy work, and also still leaves the problem that what we ship doesn't match what upstream has tested, so the result is often less reliable. IMHO the only thing that can make packaging such fine grained modules sustainable long term, is if the process could be 100% automated. I don't mean just having a tool to auto-generate a spec file. It needs to be fully automated to the extent that you just tell a robot "i need deps for this package" and it does everything automatically, except for human approval of the results of a code license audit. That of course would be a non-trivial service to build, and it still begs the question of whether its really beneficial in the long term. > > Sure, but isn't that the case for most projects that a newcomer wants > > to package, regardless of programming language? Say, somebody wants to > > package some cool new Python project for machine learning, then > > there's probably also some linear algebra package or SIMD math library > > in the dependency tree that's missing from Fedora. How is that > > different? > > I think the big difference here from my experience, and I've packaged > right across the Fedora spectrum, is that most of the python style > projects are able to use a pretty comprehensive standard library and > crypto library which helps minimise a lot of the extras I've seen in > the rust ecosystem. Python isn't that different from the Rust world to be honest. For a short while we had OpenStack in Fedora, but it wasn't sustainable to maintain its large set of python module deps, fighting between the versions Fedora already had, vs the versions that OpenStack actually wanted (and was tested against by upstream. It was never ending busy-work for the people trying to keep it working in Fedora, taking time away from more beneficial work, and delivering a system that was often broken because module version combinations we used were not tested. > > - we port projects to new versions of dependencies > > That's valuable for other projects but not Fedora, and it's not > something I am going to do personally as a rust packager. It isn't sustainable in the general case, as it requires a level of expertize / knowledge that most packagers can't be assumed to have. If they have that knowledge and the time to invest in this, they can do it directly in upstream, regardless of what Fedora packaging approach is used. > I don't think it's as bad as other ecosystems but I don't agree with > your assessment of "kind of a *good* situation", I feel there may be a > little bit of Stockholm syndrome coming into play here to be honest. I feel like Fedora is successfull in packaging huge numbers of deps across the various language ecosystems, not because it is easy, but often as a result of a relatively small number of people in the respect SIGs investing a heroic amount of effort to keep everything ticking along. That isn't long term sustainable and leaves us vulnerable to maintainers no longer being able or willing to invest their time in future, which risks huge chains of packages being orphaned and falling out of the distro. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
