On 1/15/21 9:46 AM, Kevin Fenzi wrote:
On Thu, Jan 14, 2021 at 05:44:56PM -0800, Tom Stellard wrote:
Hi,
I am considering making a proposal for an alternative buildroot (similar to
ELN) in Fedora, and I want to try to understand what Fedora resources might
be available for this.
I think you may be conflating 2 different things below...
A alternate buildroot (like eln) exists in the current koji and uses
it's builders and other resources.
My requirements are:
+ A koji server and builders for doing 'shadow' builds of ~2000 Fedora
packages. 'shadow' builds means that every time a build is made in the
official Fedora koji then one would be done for this alternative buildroot.
This is... a secondary koji setup, not an alternative buildroot. :)
OK, makes sense.
Note that back when we had secondary koji's, the shadowing scripts were
very fragile and basically took at least a few people full time to make
sure they kept building/working. It's not a 'run script and walk away'.
It's very labor instensive by someone who understands packages. :(
I have a working prototype for this that listens to Fedora and kicks off
builds on another koji server. I think what I'm doing is much easier
than a true shadow instance, because my buildroots use the main Fedora
repos as external repos.
The builds I am doing are meant to be layered on top of the Fedora repos
and are not meant to be a complete working compose on their own.
It's also been about 3+years since anyone (that I am aware of) ran one.
So, very likely old scripts would be horribly broken. ;(
+ Builders for all architectures supported by Fedora.
We don't have bunches of idle builders.
+ Developer access to the koji server using Fedora kerberos credentials.
+ Permissions to create and modify tags on the koji server for at least 3
people.
+ A DNF repo generated from these package builds that is updated once per
day.
ELN does this with odcs composes.
+ Jenkins server that can host a CI job that listens for Fedora builds and
submits builds to this alternative buildroot.
We currently run 0 jenkins servers. :) We don't really want to get into
that business, IMHO.
My questions are:
Do we have enough machine resources and people time to implement something
like this, and if not, how much additional resources would be required?
I don't think we (fedora infrastructure/cpe) wants to deploy and build
out a secondary koji hub and builders. Thats going to be a ton of work.
If you mean to do this like ELN (a seperate buildroot in the existing
koji) we need to discuss things more I think. That means we don't have
to run a seperate hub and builders, but it means the new buildroot takes
up a ton of cpu time, builder time, and disk space.
Running in the main koji instance is certainly an option, I didn't
originally suggest this, because I thought it might interfere too much
with normal Fedora work.
How and where to implement this, e.g. can we use the staging koji server or
something else?
The staging koji is to be as similar as possible to the primary one so
we can test things. If we deployed something there and it diverged or we
needed to reinstall it and destroy this it would make everyone sad.
Can we build something that is generic enough, so that other people who want
to make similar proposals would be able to easily set up their own
alternative buildroot?
I don't personally want to make this easy... it uses way too much
resources. :(
There is another alternative however:
https://mbbox-operator.readthedocs.io/en/latest/
mbbox is a openshift operater that deploys koji, db, and mbs. You still
need builders I think, but otherwise it does a lot of the heavy lifting.
This could be run in a cloud instance/cluster? That wouldn't help the
shadowing problem... but perhaps the ELN folks could share how they are
doing things?
This sounds great, I will take a look at it.
Does that answer some questions/provide more info?
Yes, thanks.
-Tom
kevin
_______________________________________________
infrastructure mailing list -- infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to infrastructure-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
_______________________________________________
infrastructure mailing list -- infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to infrastructure-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
