----- Original Message ----- > From: "nicolas mailhot" <nicolas.mailhot@xxxxxxxxxxx> > To: golang@xxxxxxxxxxxxxxxxxxxxxxx > Cc: "Development discussions related to Fedora" <devel@xxxxxxxxxxxxxxxxxxxxxxx>, "Discussion of RPM packaging > standards and practices for Fedora" <packaging@xxxxxxxxxxxxxxxxxxxxxxx> > Sent: Thursday, February 1, 2018 11:21:59 AM > Subject: Re: <DKIM> [Fedora-packaging] Re: Proposed Fedora packaging guideline: More Go packaging > > > > ----- Mail original ----- > De: "Owen Taylor" > > Hi Owen, > > > Is there a guide for Fedora packagers about how to handle unbundling for > > golang packages? The draft guidelines don't seem to go into any details. > > I don't think there is, nor that it is necessarily needed. The posted > guidelines should be sufficient technically, there are no magic I know of I > didn't document, the rest is just a lot of work (but see ↓) > > > I've looked at packaging a few golang packages unbundled, and have > > immediately run into: > > > > A) lots of unpackaged dependencies > > B) dependencies that are packaged in Fedora with different, often much > > older versions > > Yes the state of Go packages in Fedora is pretty sad right now. I wouldn't > expect anyone to be able to package anything but the most trivial app in > unbundled mode. Too many common parts are missing, when they are not missing > they are too old, trying to update an existing Go package is an exercise in > frustration (too much package-specific shell code, that is difficult to > understand and does not really work with new code versions), and trying to > update or add missing parts just reveals more breakage and work to be done. It depends (as everything) on available manpower, if you are willing to own your dependencies you can package anything and everything debundled. > > However, accepting to package complex Go apps in bundled mode is how Fedora > got to this state in the first place. In plain speak, bundling (vendoring in > Go linguo) just does not scale. You need an awful lot of manpower to audit > the hundreds of other projects each app bundles, bundling removes all the > package tooling that may help you to do so, and the result is not shared > with any other package, or with other versions of the same package, so you > get zero positive network effects. Worse, big bundled apps that do not > actually try to work in unbundled mode, do not actually test the code they > export (they are bundling, remember) so the result is toxic to small Go > packages that try to work with them. On contrary, to this state you dislike(or seems to) Fedora got because of effort of many to de-bundle Go based projects. > > So bundling parts is a direct road to bundling everything, bundling > everything is a direct road to bundling everything blindly, bundling > everything blindly is error-prone and dangerous (because upstreams are only > human and do make lots of mistakes), and pretty much removes any value > Fedora can add to end users, to other parts of Fedora that would like to > integrate with Go software, or to the upstream projects themselves (no > Fedora QA, no stream of Fedora-originated fixes, no Fedora pressure to > stabilize parts when upstream is lost in tunnel effect mode and does not > realize that it is wasting everyone's time starting with its own). > > Therefore, trying to get all this it a better state, requires the following > steps IMHO: > > 1. completely refactor our Go packaging style it's less painful to update Go > packages and they do not need a packager with deep knowledge of > package-specific shell glue. That takes documentation, and factoring out > common Go packaging functions in shared rpm code (macros and autodeps) → > that's what I posted I don't see way how it makes it less painful(you have still the whole distro issue). It makes specs more abstract and streamlined when it works, but more opaque and hard to read when its not(and you have to debug some arcane srpm/lua macros). > > 2. using the new documentation and tooling to clean up years of Fedora > technical debt, and create a new set of up-to-date Go packages that can > serve as new baseline → I have hundreds of specs that I'm waiting for step 1 > to complete to submit. They won't constitute a full baseline by themselves, > they're not all 100% done, it's too much work to do alone but working with > others requires the common macros and documentation to be merged and > adopted. This step is going to be painful I'm afraid, Fedora dug itself a > deep hole, leaving it is going to be hard. On contrary Fedora is trying to fill the hole that upstream Go projects dug them selves in to. IMNHO Go have traded any subjectively perceived "RPM/deb hell" for even deeper levels of "Go (vendor) hell". > > 3. hopefully, the result will be streamlined enough it won't be too painful > to keep up to date, having an up to date baseline will help attract new Go > packagers like you, everyone will benefit and be happy. We had to package > some ostree bits for example to create this baseline, and I'm pretty sure > ostree people within Fedora would prefer to maintain those themselves, if > the rest of the Fedora go universe didn't make it too hard Pain will not be eased in any significant way as you will still have to carefully evaluate every change so you don't break any dependent package. > > > B) more > > intimidating. It seems like to package the new package, I have to get the > > maintainer of the library to upgrade the version, > > The guidelines and automation aim at making upgrading easy, and avoid one > package or packager blocking others This is not really and automation per se. It is minor re-factoring and spec streamlining. It might or might not enable easier implementation of packaging automation. > > > and someone needs to > > rebuild everything that depends on the rebuilt library and test that the > > rebuilt packages work. > > I hope that normalizing Fedora Go packages means it will be possible to > automate QA tests such as "try to rebuild all the packages that depend on > golang(foo) every time you see the package providing golang(foo) changing, > and report what broke" > > That would be expensive computing-side but a lot less expensive and long than > expecting each Go packager to do it himself with his own means. And that is > certainly not overkill, given how lax Go projects are about maintaining API > stability. Until upstreams will be working in true CI/CD manners(when they do not make any releases). Fedora can't really fix this. Currently AFAIK we have GoFed and accompanying projects that provide you with possibility to gather such data. > > And then in case of breakage, revert or create a compat package. That's why > there is a long chapter dedicated to compat package creation in the proposed > guidelines. Compat packages do not scale. Imagine that you will have in worst case maintain 10s of different versions of same package for every release, doing security backports, etc.... JC > > Regards, > > -- > Nicolas Mailhot > _______________________________________________ > golang mailing list -- golang@xxxxxxxxxxxxxxxxxxxxxxx > To unsubscribe send an email to golang-leave@xxxxxxxxxxxxxxxxxxxxxxx > _______________________________________________ packaging mailing list -- packaging@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to packaging-leave@xxxxxxxxxxxxxxxxxxxxxxx
