On Mon, Jun 29, 2020 at 03:32:22PM +0100, Daniel P. Berrangé wrote:
> On Mon, Jun 29, 2020 at 03:58:43PM +0200, Andrea Bolognani wrote:
> > This is similar to what we already use for Dockerfiles, with one
> > key difference: while we still rely on lcitool taking care of the
> > complicated work for us, in this case we're only provided with a
> > bunch of variables and we have to do the last bit of work (that
> > is replacing them inside an existing template) ourselves.
> >
> > Signed-off-by: Andrea Bolognani <abologna@xxxxxxxxxx>
> > ---
> > ci/cirrus/refresh | 32 ++++++++++++++++++++++++++++++
> > ci/cirrus/templates/freebsd-12.yml | 25 +++++++++++++++++++++++
> > ci/cirrus/templates/macos-1015.yml | 27 +++++++++++++++++++++++++
>
> These files are largely the identical content to the same name
> files in the dir above which feels like it is easy to optimize
> away.
>
> > 3 files changed, 84 insertions(+)
> > create mode 100755 ci/cirrus/refresh
> > create mode 100644 ci/cirrus/templates/freebsd-12.yml
> > create mode 100644 ci/cirrus/templates/macos-1015.yml
> >
> > diff --git a/ci/cirrus/refresh b/ci/cirrus/refresh
> > new file mode 100755
> > index 0000000000..51deca94ac
> > --- /dev/null
> > +++ b/ci/cirrus/refresh
> > @@ -0,0 +1,32 @@
> > +#!/bin/sh
> > +
> > +if test -z "$1"
> > +then
> > + echo "syntax: $0 PATH-TO-LCITOOL"
> > + exit 1
> > +fi
> > +
> > +LCITOOL=$1
> > +
> > +if ! test -x "$LCITOOL"
> > +then
> > + echo "$LCITOOL is not executable"
> > + exit 1
> > +fi
> > +
> > +for infile in templates/*
> > +do
> > + outfile="${infile##*/}.j2"
> > + host="${outfile%%.*}"
> > +
> > + eval $("$LCITOOL" dockerfile "libvirt-$host" libvirt --variables)
> > +
> > + sed -e "s|[@]PKGS@|$PKGS|g" \
> > + -e "s|[@]CROSS_PKGS@|$CROSS_PKGS|g" \
> > + -e "s|[@]PYPI_PKGS@|$PYPI_PKGS|g" \
> > + -e "s|[@]CPAN_PKGS@|$CPAN_PKGS|g" \
> > + -e "s|[@]MAKE@|$MAKE|g" \
> > + -e "s|[@]NINJA@|$NINJA|g" \
> > + -e "s|[@]PYTHON@|$PYTHON|g" \
> > + <"$infile" >"$outfile"
> > +done
>
> I feel like this should really be reduced to just:
>
> $LCITOOL cirrusci libvirt-$host libvirt > $outfile
>
> where the 'cirrusci' command generates this content....
I agree with the idea of having a new command generating the content below, I
tend to look at this as different generator plugins/backends that we support,
so having a dedicated command makes much more sense to me.
>
> > diff --git a/ci/cirrus/templates/freebsd-12.yml b/ci/cirrus/templates/freebsd-12.yml
> > new file mode 100644
> > index 0000000000..228ea67827
> > --- /dev/null
> > +++ b/ci/cirrus/templates/freebsd-12.yml
> > @@ -0,0 +1,25 @@
> > +freebsd_instance:
> > + image_family: freebsd-12-1
> > +
> > +env:
> > + CI_REPOSITORY_URL: {{ CI_REPOSITORY_URL }}
> > + CI_COMMIT_REF_NAME: {{ CI_COMMIT_REF_NAME }}
> > + CI_COMMIT_SHA: {{ CI_COMMIT_SHA }}
> > + PKGS: @PKGS@
> > + MAKE: @MAKE@
> > + PYTHON: @PYTHON@
> > +
> > +freebsd_12_task:
> > + install_script:
> > + - pkg install -y $PKGS
> > + clone_script:
> > + - git clone --depth 100 "$CI_REPOSITORY_URL" .
> > + - git fetch origin "$CI_COMMIT_REF_NAME"
> > + - git reset --hard "$CI_COMMIT_SHA"
>
> ...down to this point.
>
>
> > + build_script:
> > + - mkdir build
> > + - cd build
> > + - ../autogen.sh --prefix=$(pwd)/install-root
> > + - $MAKE -j3
> > + - $MAKE -j3 install
> > + - $MAKE -j3 dist
>
> This part can then be stored in ci/cirrus/build.yml since it is
> common to freebsd & macos.
>
> So now in gitlab-ci.yml we can just concatenate the two into a
> temp file:
>
> cat ci/cirrus/$NAME.yml ci/cirrus/build.yml > ci/cirrus/$NAME.yml.j2
> cirrus-run ci/cirrus/$NAME.yml.j2
How about instead of storing the same build.yml in every secondary project's ci
directory we store the build.yml in libvirt-ci repo and generate the whole
thing with lcitool? I think the way the build YAML would look like for every
single subproject, we could achieve the level of similarity we have for lcitool
native builds - IOW the build job abstraction we have across projects under
guests/playbooks/build/projects could also be achieved here with cirrus, just
an idea.
Erik
