Hi,
this is the fourth version of my patch series that introduces support
for GitLab CI.
Changes compared to v3:
- Stopped using nproc(1) to figure out the number of builds jobs for
GitHub Actions and Azure Pipelines. Instead, we now continue to
use the hardcoded 10 jobs there, whereas on GitLab CI we now use
nproc. We can adapt GitHub/Azure at a later point as required, but
I don't feel comfortable doing changes there.
- Improved the linux-musl job. Namely, we now also install all
required Apache modules, which makes the Apache-based test setup
work. There is a packaging issue with the WebDAV module though, so
we now skip tests that depend on it on Alpine Linux.
I still didn't move `.gitlab-ci.yml` to `contrib/`. As Taylor argued
(and I don't disagree), moving it to `contrib/` would convey the spirit
that this is _not_ an authoritative CI pipeline setup. But I wanted to
hear other opinions first before moving it into `contrib/`.
Patrick
Patrick Steinhardt (8):
ci: reorder definitions for grouping functions
ci: make grouping setup more generic
ci: group installation of Docker dependencies
ci: split out logic to set up failed test artifacts
ci: unify setup of some environment variables
ci: squelch warnings when testing with unusable Git repo
ci: install test dependencies for linux-musl
ci: add support for GitLab CI
.gitlab-ci.yml | 53 +++++++++
ci/install-docker-dependencies.sh | 23 +++-
ci/lib.sh | 190 +++++++++++++++++++++---------
ci/print-test-failures.sh | 6 +
t/lib-httpd.sh | 17 ++-
5 files changed, 233 insertions(+), 56 deletions(-)
create mode 100644 .gitlab-ci.yml
Range-diff against v3:
1: ef44ed5c3b1 = 1: 8595fe5016a ci: reorder definitions for grouping functions
2: 77798fa7a7a = 2: 7358a943392 ci: make grouping setup more generic
3: 4542bd38dc2 = 3: 6d842592c6f ci: group installation of Docker dependencies
4: 5fdda7fd83f = 4: e15651b3f5d ci: split out logic to set up failed test artifacts
5: 6af0075fd87 ! 5: a64799b6e25 ci: unify setup of some environment variables
@@ Commit message
parts are separated. While at it, we also perform some additional small
improvements:
- - We use nproc instead of a hardcoded count of jobs for make and
- prove. This ensures that the number of concurrent processes adapts
- to the host automatically.
-
- We now always pass `--state=failed,slow,save` via GIT_PROVE_OPTS.
It doesn't hurt on platforms where we don't persist the state, so
this further reduces boilerplate.
@@ ci/lib.sh: then
exit 1
fi
-+MAKEFLAGS="$MAKEFLAGS --jobs=$(nproc)"
-+GIT_PROVE_OPTS="--timer --jobs $(nproc) --state=failed,slow,save"
++MAKEFLAGS="$MAKEFLAGS --jobs=10"
++GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
+
+GIT_TEST_OPTS="$GIT_TEST_OPTS --verbose-log -x"
+if test windows = "$CI_OS_NAME"
6: 78d863bf24e = 6: f7d2a8666fe ci: squelch warnings when testing with unusable Git repo
7: f150d61a1ce ! 7: 9b43b0d90e3 ci: install test dependencies for linux-musl
@@ Commit message
available, all Subversion-related tests are skipped without the
SVN::Core Perl library anyway.
- Furthermore, in order to make the Apache-based tests work, this commit
- also adds the Alpine-specific modules path of it to the list of known
- paths.
+ The Apache2-based tests require a bit more care though. For one, the
+ module path is different on Alpine Linux, which requires us to add it to
+ the list of known module paths to detect it. But second, the WebDAV
+ module on Alpine Linux is broken because it does not bundle the default
+ database backend [1]. We thus need to skip the WebDAV-based tests on
+ Alpine Linux for now.
+
+ [1]: https://gitlab.alpinelinux.org/alpine/aports/-/issues/13112
Signed-off-by: Patrick Steinhardt <ps@xxxxxx>
@@ ci/install-docker-dependencies.sh: linux32)
apk add --update build-base curl-dev openssl-dev expat-dev gettext \
- pcre2-dev python3 musl-libintl perl-utils ncurses >/dev/null
+ pcre2-dev python3 musl-libintl perl-utils ncurses \
-+ apache2 bash cvs gnupg perl-cgi perl-dbd-sqlite >/dev/null
++ apache2 apache2-http2 apache2-proxy apache2-ssl apache2-webdav apr-util-dbd_sqlite3 \
++ bash cvs gnupg perl-cgi perl-dbd-sqlite >/dev/null
;;
pedantic)
dnf -yq update >/dev/null &&
@@ t/lib-httpd.sh: for DEFAULT_HTTPD_MODULE_PATH in '/usr/libexec/apache2' \
do
if test -d "$DEFAULT_HTTPD_MODULE_PATH"
then
+@@ t/lib-httpd.sh: else
+ "Could not identify web server at '$LIB_HTTPD_PATH'"
+ fi
+
++if test -n "$LIB_HTTPD_DAV" && test -f /etc/os-release
++then
++ case "$(grep "^ID=" /etc/os-release | cut -d= -f2-)" in
++ alpine)
++ # The WebDAV module in Alpine Linux is broken at least up to
++ # Alpine v3.16 as the default DBM driver is missing.
++ #
++ # https://gitlab.alpinelinux.org/alpine/aports/-/issues/13112
++ test_skip_or_die GIT_TEST_HTTPD \
++ "Apache WebDAV module does not have default DBM backend driver"
++ ;;
++ esac
++fi
++
+ install_script () {
+ write_script "$HTTPD_ROOT_PATH/$1" <"$TEST_PATH/$1"
+ }
8: 5272d66d9f1 ! 8: f3f2c98a0dc ci: add support for GitLab CI
@@ ci/install-docker-dependencies.sh: linux32)
- apk add --update build-base curl-dev openssl-dev expat-dev gettext \
+ apk add --update shadow sudo build-base curl-dev openssl-dev expat-dev gettext \
pcre2-dev python3 musl-libintl perl-utils ncurses \
- apache2 bash cvs gnupg perl-cgi perl-dbd-sqlite >/dev/null
+ apache2 apache2-http2 apache2-proxy apache2-ssl apache2-webdav apr-util-dbd_sqlite3 \
+ bash cvs gnupg perl-cgi perl-dbd-sqlite >/dev/null
;;
+linux-*)
+ apt update -q &&
@@ ci/lib.sh: then
else
begin_group () { :; }
end_group () { :; }
+@@ ci/lib.sh: then
+ cache_dir="$HOME/test-cache/$SYSTEM_PHASENAME"
+
+ GIT_TEST_OPTS="--write-junit-xml"
++ JOBS=10
+ elif test true = "$GITHUB_ACTIONS"
+ then
+ CI_TYPE=github-actions
@@ ci/lib.sh: then
cache_dir="$HOME/none"
GIT_TEST_OPTS="--github-workflow-markup"
++ JOBS=10
+elif test true = "$GITLAB_CI"
+then
+ CI_TYPE=gitlab-ci
@@ ci/lib.sh: then
+ cache_dir="$HOME/none"
+
+ runs_on_pool=$(echo "$CI_JOB_IMAGE" | tr : -)
++ JOBS=$(nproc)
else
echo "Could not identify CI type" >&2
env >&2
+ exit 1
+ fi
+
+-MAKEFLAGS="$MAKEFLAGS --jobs=10"
+-GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
++MAKEFLAGS="$MAKEFLAGS --jobs=${JOBS}"
++GIT_PROVE_OPTS="--timer --jobs ${JOBS} --state=failed,slow,save"
+
+ GIT_TEST_OPTS="$GIT_TEST_OPTS --verbose-log -x"
+ if test windows = "$CI_OS_NAME"
## ci/print-test-failures.sh ##
@@ ci/print-test-failures.sh: do
--
2.42.0
Attachment:
signature.asc
Description: PGP signature
