Right now we're dividing the jobs into three stages: prebuild, which includes DCO checking as well as building artifacts such as the website, and native_build/cross_build, which do exactly what you'd expect based on their names. This organization is nice from the logical point of view, but results in poor utilization of the available CI resources: in particular, the fact that cross_build jobs can only start after all native_build jobs have finished means that if even a single one of the latter takes a bit longer the pipeline will stall, and with native builds taking anywhere from less than 10 minutes to more than 20, this happens all the time. Building artifacts in a separate pipeline stage also doesn't have any advantages, and only delays further stages by a couple of minutes. The only job that really makes sense in its own stage is the DCO check, because it's extremely fast (less than 1 minute) and, if that fails, we can avoid kicking off all other jobs. Reducing the number of stages results in significant speedups: specifically, going from three stages to two stages reduces the overall completion time for a full CI pipeline from ~45 minutes[1] to ~30 minutes[2]. [1] https://gitlab.com/abologna/libvirt/-/pipelines/154751893 [2] https://gitlab.com/abologna/libvirt/-/pipelines/154771173 Signed-off-by: Andrea Bolognani <abologna@xxxxxxxxxx> --- .gitlab-ci.yml | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 7a8142b506..8d9313e415 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -2,9 +2,8 @@ variables: GIT_DEPTH: 100 stages: - - prebuild - - native_build - - cross_build + - sanity_checks + - builds .script_variables: &script_variables | export MAKEFLAGS="-j$(getconf _NPROCESSORS_ONLN)" @@ -17,7 +16,7 @@ stages: # Default native build jobs that are always run .native_build_default_job_template: &native_build_default_job_definition - stage: native_build + stage: builds cache: paths: - ccache/ @@ -42,7 +41,7 @@ stages: # system other than Linux. These jobs will only run if the required # setup has been performed on the GitLab account (see ci/README.rst). .cirrus_build_default_job_template: &cirrus_build_default_job_definition - stage: native_build + stage: builds image: registry.gitlab.com/libvirt/libvirt-ci/cirrus-run:master script: - cirrus-run ci/cirrus/$NAME.yml.j2 @@ -64,7 +63,7 @@ stages: # Default cross build jobs that are always run .cross_build_default_job_template: &cross_build_default_job_definition - stage: cross_build + stage: builds cache: paths: - ccache/ @@ -194,7 +193,7 @@ mingw64-fedora-rawhide: # be deployed to the web root: # https://gitlab.com/libvirt/libvirt/-/jobs/artifacts/master/download?job=website website: - stage: prebuild + stage: builds before_script: - *script_variables script: @@ -216,7 +215,7 @@ website: codestyle: - stage: prebuild + stage: builds before_script: - *script_variables script: @@ -231,7 +230,7 @@ codestyle: # for translation usage: # https://gitlab.com/libvirt/libvirt/-/jobs/artifacts/master/download?job=potfile potfile: - stage: prebuild + stage: builds only: - master before_script: @@ -259,7 +258,7 @@ potfile: # this test on developer's personal forks from which # merge requests are submitted check-dco: - stage: prebuild + stage: sanity_checks image: registry.gitlab.com/libvirt/libvirt-ci/check-dco:master script: - /check-dco -- 2.25.4
