On Wed, Jun 10, 2020 at 05:34:13PM +0200, Andrea Bolognani wrote: > Instead of using pre-built containers hosted on Quay, build > containers as part of the GitLab CI pipeline and upload them to the > GitLab container registry for later use. > > This will not significantly slow down builds, because containers are > only rebuilt when the corresponding Dockerfile has been modified. > > Signed-off-by: Andrea Bolognani <abologna@xxxxxxxxxx> > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml > index ba31afd205..f79f930e66 100644 > --- a/.gitlab-ci.yml > +++ b/.gitlab-ci.yml > @@ -3,6 +3,7 @@ variables: > > stages: > - sanity_checks > + - containers > - prebuild > - native_build > - cross_build > @@ -16,10 +17,43 @@ stages: > > # Common templates > > +# Containers that are necessary for the default pipeline > +.container_default_job_template: &container_default_job_definition > + image: docker:stable > + stage: containers > + services: > + - docker:dind > + before_script: > + - export TAG="$CI_REGISTRY_IMAGE/ci-$NAME:$CI_COMMIT_REF_SLUG" > + - export COMMON_TAG="$CI_REGISTRY/libvirt/libvirt/ci-$NAME:master" This is different to what we've done on all the other repos. I originally used this, but noted that it results in a ever growing set of tags being published in the container registry, as users will have a new branch name for every piece of work. It also means you'll never a get a cache hit from the user's registry across feature branches, though that is mitigated to by fact that we'll consider the global cache too I guess. > + - docker info > + - docker login registry.gitlab.com -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" > + script: > + - docker pull "$TAG" || docker pull "$COMMON_TAG" || true > + - docker build --cache-from "$TAG" --cache-from "$COMMON_TAG" --tag "$TAG" -f "ci/containers/ci-$NAME.Dockerfile" ci/containers > + - docker push "$TAG" > + after_script: > + - docker logout > + > +# Containers that are only needed for the full pipeline > +.container_extra_job_template: &container_extra_job_definition > + <<: *container_default_job_definition > + only: > + - master > + - /^ci-full-.*$/ > + > +# We build many containers which can be useful to debug problems but are not > +# needed for the pipeline itself to complete: those sometimes fail, and when > +# that happens it's mostly because of temporary issues with Debian sid. We > +# don't want those failures to affect the overall pipeline status > +.container_optional_job_template: &container_optional_job_definition > + <<: *container_extra_job_definition > + allow_failure: true > + > # Default native build jobs that are always run > .native_build_default_job_template: &native_build_default_job_definition > stage: native_build > - image: quay.io/libvirt/buildenv-libvirt-$NAME:latest > + image: $CI_REGISTRY_IMAGE/ci-$NAME:$CI_COMMIT_REF_SLUG > cache: > paths: > - ccache/ > @@ -67,7 +101,7 @@ stages: > # Default cross build jobs that are always run > .cross_build_default_job_template: &cross_build_default_job_definition > stage: cross_build > - image: quay.io/libvirt/buildenv-libvirt-$NAME-cross-$CROSS:latest > + image: $CI_REGISTRY_IMAGE/ci-$NAME-cross-$CROSS:$CI_COMMIT_REF_SLUG > cache: > paths: > - ccache/ > @@ -89,6 +123,212 @@ stages: > - /^ci-full-.*$/ > > > +# Native container build jobs > + > +x64-centos-7-container: > + <<: *container_default_job_definition > + variables: > + NAME: centos-7 > + > +x64-centos-8-container: > + <<: *container_default_job_definition > + variables: > + NAME: centos-8 > + > +x64-centos-stream-container: > + <<: *container_extra_job_definition > + variables: > + NAME: centos-stream > + > +x64-debian-9-container: > + <<: *container_extra_job_definition > + variables: > + NAME: debian-9 > + > +x64-debian-10-container: > + <<: *container_default_job_definition > + variables: > + NAME: debian-10 > + > +x64-debian-sid-container: > + <<: *container_extra_job_definition > + variables: > + NAME: debian-sid > + > +x64-fedora-31-container: > + <<: *container_extra_job_definition > + variables: > + NAME: fedora-31 > + > +x64-fedora-32-container: > + <<: *container_default_job_definition > + variables: > + NAME: fedora-32 > + > +x64-fedora-rawhide-container: > + <<: *container_default_job_definition > + variables: > + NAME: fedora-rawhide > + > +x64-opensuse-151-container: > + <<: *container_default_job_definition > + variables: > + NAME: opensuse-151 > + > +x64-ubuntu-1804-container: > + <<: *container_extra_job_definition > + variables: > + NAME: ubuntu-1804 > + > +x64-ubuntu-2004-container: > + <<: *container_default_job_definition > + variables: > + NAME: ubuntu-2004 > + > + > +# Cross-build containers build jobs > + > +aarch64-debian-9-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-9-cross-aarch64 > + > +armv6l-debian-9-container: > + <<: *container_extra_job_definition > + variables: > + NAME: debian-9-cross-armv6l > + > +armv7l-debian-9-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-9-cross-armv7l > + > +mips-debian-9-container: > + <<: *container_extra_job_definition > + variables: > + NAME: debian-9-cross-mips > + > +mips64el-debian-9-container: > + <<: *container_extra_job_definition > + variables: > + NAME: debian-9-cross-mips64el > + > +mipsel-debian-9-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-9-cross-mipsel > + > +ppc64le-debian-9-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-9-cross-ppc64le > + > +s390x-debian-9-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-9-cross-s390x > + > +aarch64-debian-10-container: > + <<: *container_extra_job_definition > + variables: > + NAME: debian-10-cross-aarch64 > + > +armv6l-debian-10-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-10-cross-armv6l > + > +armv7-ldebian-10-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-10-cross-armv7l > + > +i686-debian-10-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-10-cross-i686 > + > +mips-debian-10-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-10-cross-mips > + > +mips64el-debian-10-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-10-cross-mips64el > + > +mipsel-debian-10-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-10-cross-mipsel > + > +ppc64le-debian-10-container: > + <<: *container_extra_job_definition > + variables: > + NAME: debian-10-cross-ppc64le > + > +s390x-debian-10-container: > + <<: *container_default_job_definition > + variables: > + NAME: debian-10-cross-s390x > + > +aarch64-debian-sid-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-sid-cross-aarch64 > + > +armv6l-debian-sid-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-sid-cross-armv6l > + > +armv7-ldebian-sid-container: > + <<: *container_default_job_definition > + variables: > + NAME: debian-sid-cross-armv7l > + > +i686-debian-sid-container: > + <<: *container_extra_job_definition > + variables: > + NAME: debian-sid-cross-i686 > + > +mips-debian-sid-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-sid-cross-mips > + > +mips64el-debian-sid-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-sid-cross-mips64el > + > +mipsel-debian-sid-container: > + <<: *container_extra_job_definition > + variables: > + NAME: debian-sid-cross-mipsel > + > +ppc64le-debian-sid-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-sid-cross-ppc64le > + > +s390x-debian-sid-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-sid-cross-s390x > + > +mingw32-fedora-rawhide-container: > + <<: *container_default_job_definition > + variables: > + NAME: fedora-rawhide-cross-mingw32 > + > +mingw64-fedora-rawhide-container: > + <<: *container_default_job_definition > + variables: > + NAME: fedora-rawhide-cross-mingw64 > + > + > # Native architecture build + test jobs > > x64-debian-9: > @@ -116,6 +356,11 @@ x64-centos-8: > variables: > NAME: centos-8 > > +x64-centos-stream: > + <<: *native_build_extra_job_definition > + variables: > + NAME: centos-stream > + > x64-fedora-31: > <<: *native_build_extra_job_definition > variables: > @@ -231,6 +476,7 @@ mingw64-fedora-rawhide: > # https://gitlab.com/libvirt/libvirt/-/jobs/artifacts/master/download?job=website > website: > stage: prebuild > + image: $CI_REGISTRY_IMAGE/ci-centos-8:$CI_COMMIT_REF_SLUG > before_script: > - *script_variables > script: > @@ -241,7 +487,6 @@ website: > - $MAKE -C docs install > - cd .. > - mv vroot/share/doc/libvirt/html/ website > - image: quay.io/libvirt/buildenv-libvirt-centos-8:latest > artifacts: > expose_as: 'Website' > name: 'website' > @@ -253,6 +498,7 @@ website: > > codestyle: > stage: prebuild > + image: $CI_REGISTRY_IMAGE/ci-centos-8:$CI_COMMIT_REF_SLUG > before_script: > - *script_variables > script: > @@ -260,7 +506,6 @@ codestyle: > - cd build > - ../autogen.sh || (cat config.log && exit 1) > - $MAKE syntax-check > - image: quay.io/libvirt/buildenv-libvirt-centos-8:latest > > > # This artifact published by this job is downloaded to push to Weblate > @@ -268,6 +513,7 @@ codestyle: > # https://gitlab.com/libvirt/libvirt/-/jobs/artifacts/master/download?job=potfile > potfile: > stage: prebuild > + image: $CI_REGISTRY_IMAGE/ci-centos-8:$CI_COMMIT_REF_SLUG > only: > - master > before_script: > @@ -280,7 +526,6 @@ potfile: > - $MAKE -C po libvirt.pot > - cd .. > - cp po/libvirt.pot libvirt.pot > - image: quay.io/libvirt/buildenv-libvirt-centos-8:latest > artifacts: > expose_as: 'Potfile' > name: 'potfile' > diff --git a/ci/containers/README.rst b/ci/containers/README.rst > new file mode 100644 > index 0000000000..530897e311 > --- /dev/null > +++ b/ci/containers/README.rst > @@ -0,0 +1,14 @@ > +CI job assets > +============= > + > +This directory contains assets used in the automated CI jobs, most > +notably the Dockerfiles used to build container images in which the > +CI jobs then run. > + > +The ``refresh`` script is used to re-create the Dockerfiles using the > +``lcitool`` command that is provided by repo > +https://gitlab.com/libvirt/libvirt-ci > + > +The containers are built during the CI process and cached in the GitLab > +container registry of the project doing the build. The cached containers > +can be deleted at any time and will be correctly rebuilt. > > [... imagine lots and lots of Dockerfiles here ...] > > diff --git a/ci/containers/refresh b/ci/containers/refresh > new file mode 100755 > index 0000000000..8c00363ae1 > --- /dev/null > +++ b/ci/containers/refresh > @@ -0,0 +1,43 @@ > +#!/bin/sh > + > +if test -z "$1" > +then > + echo "syntax: $0 PATH-TO-LCITOOL" > + exit 1 > +fi > + > +LCITOOL=$1 > + > +if ! test -x "$LCITOOL" > +then > + echo "$LCITOOL is not executable" > + exit 1 > +fi > + > +HOSTS=$($LCITOOL hosts | grep -v freebsd) > + > +for host in $HOSTS > +do > + name=${host#libvirt-} > + > + case "$name" in > + fedora-rawhide) > + for cross in mingw32 mingw64 > + do > + $LCITOOL dockerfile $host libvirt --cross $cross >ci-$name-cross-$cross.Dockerfile > + done > + ;; > + debian-*) > + for cross in aarch64 armv6l armv7l i686 mips mips64el mipsel ppc64le s390x > + do > + if test "$name" = "debian-9" && test "$cross" = "i686" > + then > + continue > + fi > + $LCITOOL dockerfile $host libvirt --cross $cross >ci-$name-cross-$cross.Dockerfile > + done > + ;; > + esac > + > + $LCITOOL dockerfile $host libvirt >ci-$name.Dockerfile In all the other projects we've just used $host.Dockerfile. If we really want to eliminate the "libvirt-" prefix, then it is probably better to modify lcitool to remove this prefix. It is a pretty pointless name prefix considering the targets are used for non-libvirt projects too. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
