On Mon, Feb 06, 2023 at 03:45:12PM +0000, Daniel P. Berrangé wrote: > On Mon, Feb 06, 2023 at 02:52:57PM +0100, Erik Skultety wrote: > I appreciate the goal of being able to run CI commands locally, but > I'm not feeling this approach is a net win. I'd much rather just > having developers invoke the meson command directly, which is not > that hard. > > If we do really want to provide something that 100% mirrors the > gitlab CI job commands though, I'm even more convinced now that > we should be using the .gitlab-ci.yml as the canonical source. > > Since I last mentioned that idea, I found out that this should > be something we can achieve via the gitlab runner 'exec' command. I wonder if we've been thinking about this from the wrong angle. We've been looking at the problem of "we have a gitlab-ci.yml" and we want to be able to locally run the jobs it defines. This creates us the problem of interpreting the gitlab-ci.yml file, which is very hard. 95% of what we define in the gitlab-ci.yml file comes in via the includes of ci/gitlab.yml which is entirely auto-generated from our ci/manifest.yml file. IOW, can we rephase the problem to "we have a ci/manifest.yml" and we want to locally run the jobs it implies. We already have the code for parsing the ci/manifest.yml file, and so (mostly) know what jobs we expect to have. What we're missing is the project specific build rules which still live in the hand crafted .gitlab-ci.yml. If we could get the project specific build rules to be expressed in the ci/manifest.yml file, then we have all the info we need to be able to run jobs locally. We could then make even the root .gitlab-ci.yml be auto-generated, which would be nice. If we have the project build rules in ci/manifest.yml, we ahve lots of flexibility. We can easily trigger the builds in containers or VMs, or the local host, or whatever. 'lcitool' already has the command for runing builds in VMs, but that relies on build rules we defined separately in libvirt-ci.git and we've somewhat neglected the VM build logic since adopting containers. Essentially ci/manifest.yml would become a fully self-contained canonical representation of any info needed to run builds, and be independant of gitlab CI, or any other CI framework we might like to plug into in the future. So 'lcitool' can be the entrypoint for triggering builds that 100% match what gets run in gitlab Yes, this only addresses the core build/unittest side of things. The integration test suite is still a separate task to address. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
