On Tue, Jan 11, 2022 at 01:02:59PM +0100, Lumír Balhar wrote: > Hello. > > My name is Lumír and I maintain a few layered container images in > Fedora/RHEL/upstream related to Python and S2I (source to image) platform - > namely python3, s2i-core, and s2i-base. > > TL;DR: We (maintainers of layered container images for languages, runtimes > and databases) are considering migrating our images from Fedora > infrastructure and Fedora containers registry somewhere else. Below are the > reasons for that. If you think it’s a bad idea, let us know why. If you are > a user of those images, let us know as well so we can stay in touch with > you. > > Forgive me for my wording but I’m really sad about the current situation. yeah, it is sad. ;( > I have to honestly admit that I’m disappointed about the lack of attention > containers in Fedora are getting. Maintaining them is painful, > infrastructure is unstable and unreliable and there was literally no > movement forward in recent years. Let me summarize it. > > Container SIG is inactive. See important issues opened for years with no > movement further: https://pagure.io/ContainerSIG/container-sig/issues > > Even a simple task like merging rawhide branch to f35 for example is far > from smooth because they are never the same, because manual changes are > required for each branch: > https://pagure.io/ContainerSIG/container-sig/issue/16 (3 years old, no > reply) > > S2I container images depend on each other (python3 -> s2i-base -> s2i-core) > but there is no automation for chain rebuilds so if I want to update > s2i-core image (to fix a CVE or something like that) and all dependant > images, it takes weeks because I have to rebuild them one by one and let > them one by one pass through Bodhi updates until I can rebuild next one in > the queue. And it’s really frustrating given the insufficient stability of > the infra. There seems to be a way how to make this faster, but nobody > knows, how it works: https://pagure.io/ContainerSIG/container-sig/issue/52 > and https://github.com/containerbuildsystem/atomic-reactor/issues/1730 > > The same applies also if the fix is done on the RPM level because there are > no freshmaker nor automated rebuilds. > > The branching structure does not really fit container needs. I maintain the > s2i Python container and I don’t really care whether it’s based on Fedora 34 > or 35 (both have Python 3.10 as the main one) so I’d ideally have one branch > for Python 3.10, one for 3.9 etc. instead of two separated branches (f34, > f35) where the Python is the same version. > https://pagure.io/ContainerSIG/container-sig/issue/12 (3 years old) Yeah, I think all the above is kind of a consequence of there not being a active containers SIG. Infrastructure and releng doesn't really have cycles or interest in driving containers forward, so it needs some other folks attention. ;( > An example of unstable infrastructure: I’m not able to build an image for > three weeks now and a response I got is to report the problem upstream: > https://pagure.io/fedora-infrastructure/issue/10437 > I know that nothing is 100% reliable but if you take a look at these issues, > there are a lot of them: https://pagure.io/fedora-infrastructure/issues?search_pattern=container&status=Closed Do note that that issue was filed on December 21st. Myself and much of the infrastructure team were off on holidays. I only got back yesterday and there's a ton of things to do, but I did try and look into this some. The reason there's so many issues I think is because the entire pipeline is just way overkill and complex for just building containers. Basically we have to have koji, koji builders, and 2 openshift clusters (one for x86 and one for aarch64) to build containers, they have all kinds of fragile handoffs between each other as well. > And it seems that the packages in the infrastructure are outdated a lot: > https://github.com/containerbuildsystem/atomic-reactor/issues/1742#issuecomment-1009279161 To clarify, thats installed in the build container... so, _fedora_ packages are outdated. ;( I have seen atomic-reactor being old, but I've been afraid to update it since I don't know the pipeline. Co-maintainers of those packages in Fedora would sure help out. > If we migrate our container images to some other registry (e.g. a common > fedora space on quay.io), we’ll be able to rebuild them after every merged > PR or every week, do chain rebuilds and push them to the registry directly > from Github CI. This will make our lives significantly easier, and in turn > our images better. We are looking into dropping out registry and moving to quay.io ourselves. ;) > Are there any benefits to using Fedora infrastructure and Fedora containers > registry which I don’t see that we should consider? I'm not sure... hopefully folks who know the pipeline will chime in here. kevin
Attachment:
signature.asc
Description: PGP signature
_______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
