On Wed, 2020-06-17 at 14:35 +0100, Daniel P. Berrangé wrote: > On Wed, Jun 17, 2020 at 12:54:56PM +0200, Andrea Bolognani wrote: > > On Thu, 2020-06-11 at 17:42 +0100, Daniel P. Berrangé wrote: > > > +.script_variables: &script_variables | > > > + export MAKEFLAGS="-j$(getconf _NPROCESSORS_ONLN)" > > > + export VROOT="$SCRATCH_DIR/vroot" > > > + export CCACHE_BASEDIR="$(pwd)" > > > + export CCACHE_DIR="$CCACHE_BASEDIR/ccache" > > > + export CCACHE_MAXSIZE="500M" > > > + export PATH="$CCACHE_WRAPPERSDIR:$VROOT/bin:$PATH" > > > + export SCRATCH_DIR="/tmp/scratch" > > > + export PKG_CONFIG_PATH="$VROOT/lib/pkgconfig" > > > > You need to define $SCRATCH_DIR before $VROOT, otherwise the latter > > will get the wrong value. > > > > I also just realized that the way we set $CCACHE_BASEDIR might not > > work for the libvirt build that we perform as a prerequisite, and > > There's no problem - $(pwd) expands at the time the variable is > defined. The problem with $CCACHE_BASEDIR being set to $PWD is that the build that we do for libvirt happens *outside* of that directory, which I think messes up with the path rewriting and might result in libvirt builds not taking proper advantage of the cache. It's all a bit confusing, to be honest, so I could have gotten it wrong, but IIUC the idea is that if you have multiple built trees for the same project you'd be able to do something like $ cd ~/build/tree/one $ export CCACHE_BASEDIR=$PWD $ make $ cd ~/build/tree/two $ export CCACHE_BASEDIR=$PWD $ make and, thanks to $CCACHE_BASEDIR, ccache will be able to figure out that some of the files are perfect matches and will thus achieve better hit rates. This is not our case for a few reasons: * we only build each project once per container; * we set $CCACHE_BASEDIR once instead of once per build directory; * we build libvirt outside of $CCACHE_BASEDIR. So basically our current use of $CCACHE_BASEDIR is likely useless if not even somewhat harmful, and we should just get rid of it. > > since we use the same paths across builds anyway there doesn't seem > > to be a point in setting it. So I suggest we have > > > > export SCRATCH_DIR="/tmp/scratch" > > export VROOT="$SCRATCH_DIR/vroot" > > export CCACHE_DIR="$SCRATCH_DIR/ccache" > > The ccache dir has to be a subdir of the source checkout for > gitlab to cache it between jobs. Right! I think you told me about this already, but then I forgot about it O:-) Keeping that in mind, we should have export CCACHE_DIR="$PWD/ccache" then. > > > + script: > > > + - mkdir build > > > + - cd build > > > + - ../autogen.sh --prefix="$VROOT" > > > + - $MAKE install > > > + - $MAKE dist > > > > Do we want distcheck here, or is the combination of dist plus > > building the RPM package (which effectively runs check inside the > > generated tarball) good enough? > > I think its good enough. Also I want to switch to meson for glib > asap. Okay. Be aware of the fact that, with the switch to Meson, we're going to need something like https://gitlab.com/libosinfo/libosinfo/-/blob/master/.gitlab-ci.yml#L6-13 because Meson, unlike autotools, is properly multiarch-aware. In fact, a better solution would probably be to bake the multilib string into the Dockerfile directly, similarly to what we already do for other multiarch-related information, instead of figuring it out dynamically at build time. I haven't had the time to look into implementing that yet, unfortunatley, but maybe we can get that in before libvirt-glib moves off autotools. -- Andrea Bolognani / Red Hat / Virtualization
