Hi Nicolas,
I'm embarrassed to admit that before I sent my mail I carefully read over ... the old PackageDrafts/Go :-( My only excuse is that it was in my browser history.Having actually read the relevant parts of "More Go Packaging", the explanation of compat packages and notification procedures does make the intended operation clearer, though the social and technical barrier to a packager new to Fedora will still be high if packaging their target package requires creating a compat package and fixing multiple other packages. As you say, once Fedora packaging catches up to the modern state, things will be better, because the onus won't be on the new packager to do the work of updating old libraries.
I still worry that Fedora is not big enough to move the status-quo in the Go world - to get the point where Go programs require github.com/foo/bar >= 1.2.3 and actually have been tested with a multiple versions in that range, not exactly the one vendored version shipped with the program. Sp we might end up with lots of Fedora-specific bugs that the upstream doesn't care about. But if the Fedora Go community can get some size and momentum and start pushing fixes upstream, that will certainly be a positive force for reducing Go programs stuck on random old versions of dependencies!
I haven't had time to read through the entire proposal, but it certainly looks like a major step forward!
Owen
On Thu, Feb 1, 2018 at 5:21 AM, <nicolas.mailhot@xxxxxxxxxxx> wrote:
----- Mail original -----
De: "Owen Taylor"
Hi Owen,
> Is there a guide for Fedora packagers about how to handle unbundling for
> golang packages? The draft guidelines don't seem to go into any details.
I don't think there is, nor that it is necessarily needed. The posted guidelines should be sufficient technically, there are no magic I know of I didn't document, the rest is just a lot of work (but see ↓)
> I've looked at packaging a few golang packages unbundled, and have
> immediately run into:
>
> A) lots of unpackaged dependencies
> B) dependencies that are packaged in Fedora with different, often much
> older versions
Yes the state of Go packages in Fedora is pretty sad right now. I wouldn't expect anyone to be able to package anything but the most trivial app in unbundled mode. Too many common parts are missing, when they are not missing they are too old, trying to update an existing Go package is an exercise in frustration (too much package-specific shell code, that is difficult to understand and does not really work with new code versions), and trying to update or add missing parts just reveals more breakage and work to be done.
However, accepting to package complex Go apps in bundled mode is how Fedora got to this state in the first place. In plain speak, bundling (vendoring in Go linguo) just does not scale. You need an awful lot of manpower to audit the hundreds of other projects each app bundles, bundling removes all the package tooling that may help you to do so, and the result is not shared with any other package, or with other versions of the same package, so you get zero positive network effects. Worse, big bundled apps that do not actually try to work in unbundled mode, do not actually test the code they export (they are bundling, remember) so the result is toxic to small Go packages that try to work with them.
So bundling parts is a direct road to bundling everything, bundling everything is a direct road to bundling everything blindly, bundling everything blindly is error-prone and dangerous (because upstreams are only human and do make lots of mistakes), and pretty much removes any value Fedora can add to end users, to other parts of Fedora that would like to integrate with Go software, or to the upstream projects themselves (no Fedora QA, no stream of Fedora-originated fixes, no Fedora pressure to stabilize parts when upstream is lost in tunnel effect mode and does not realize that it is wasting everyone's time starting with its own).
Therefore, trying to get all this it a better state, requires the following steps IMHO:
1. completely refactor our Go packaging style it's less painful to update Go packages and they do not need a packager with deep knowledge of package-specific shell glue. That takes documentation, and factoring out common Go packaging functions in shared rpm code (macros and autodeps) → that's what I posted
2. using the new documentation and tooling to clean up years of Fedora technical debt, and create a new set of up-to-date Go packages that can serve as new baseline → I have hundreds of specs that I'm waiting for step 1 to complete to submit. They won't constitute a full baseline by themselves, they're not all 100% done, it's too much work to do alone but working with others requires the common macros and documentation to be merged and adopted. This step is going to be painful I'm afraid, Fedora dug itself a deep hole, leaving it is going to be hard.
3. hopefully, the result will be streamlined enough it won't be too painful to keep up to date, having an up to date baseline will help attract new Go packagers like you, everyone will benefit and be happy. We had to package some ostree bits for example to create this baseline, and I'm pretty sure ostree people within Fedora would prefer to maintain those themselves, if the rest of the Fedora go universe didn't make it too hard
> B) more
> intimidating. It seems like to package the new package, I have to get the
> maintainer of the library to upgrade the version,
The guidelines and automation aim at making upgrading easy, and avoid one package or packager blocking others
> and someone needs to
> rebuild everything that depends on the rebuilt library and test that the
> rebuilt packages work.
I hope that normalizing Fedora Go packages means it will be possible to automate QA tests such as "try to rebuild all the packages that depend on golang(foo) every time you see the package providing golang(foo) changing, and report what broke"
That would be expensive computing-side but a lot less expensive and long than expecting each Go packager to do it himself with his own means. And that is certainly not overkill, given how lax Go projects are about maintaining API stability.
And then in case of breakage, revert or create a compat package. That's why there is a long chapter dedicated to compat package creation in the proposed guidelines.
Regards,
--
Nicolas Mailhot
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@lists.fedoraproject.org
_______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
