Re: [libvirt PATCH 4/4] gitlab-ci: Introduce a new test 'integration' pipeline stage

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Mon, Jan 31, 2022 at 07:01:01PM +0100, Erik Skultety wrote:
> Create an integration child pipeline in this stage which will trigger a
> multi-project CI build of Perl bindings which are required by the TCK
> test suite.
> In general, this stage will install all the necessary build artifacts
> and configure logging on the worker node prior to executing the actual
> test suite. In case of a failure, libvirt and Avocado logs are saved
> and published as artifacts.
> 
> Signed-off-by: Erik Skultety <eskultet@xxxxxxxxxx>
> ---
>  .gitlab-ci-integration.yml | 116 +++++++++++++++++++++++++++++++++++++
>  .gitlab-ci.yml             |  14 +++++
>  2 files changed, 130 insertions(+)
>  create mode 100644 .gitlab-ci-integration.yml
> 
> diff --git a/.gitlab-ci-integration.yml b/.gitlab-ci-integration.yml
> new file mode 100644
> index 0000000000..cabefc5166
> --- /dev/null
> +++ b/.gitlab-ci-integration.yml
> @@ -0,0 +1,116 @@
> +stages:
> +  - bindings
> +  - integration
> +
> +.tests:
> +  stage: integration
> +  before_script:
> +    - mkdir "$SCRATCH_DIR"
> +    - sudo dnf install -y libvirt-rpms/* libvirt-perl-rpms/*
> +    - sudo pip3 install --prefix=/usr avocado-framework
> +    - source /etc/os-release  # in order to query the vendor-provided variables
> +    - if test "$ID" == "centos" && test "$VERSION_ID" -lt 9 ||
> +         test "$ID" == "fedora" && test "$VERSION_ID" -lt 35;
> +      then
> +        DAEMONS="libvirtd virtlogd virtlockd";
> +      else
> +        DAEMONS="virtproxyd virtqemud virtinterfaced virtsecretd virtstoraged virtnwfilterd virtnodedevd virtlogd virtlockd";
> +      fi
> +    - for daemon in $DAEMONS;
> +      do
> +        sudo sed -Ei "s/^(#)?(log_outputs=).*/\2'1:file:\/var\/log\/libvirt\/${daemon}.log'/" /etc/libvirt/${daemon}.conf;
> +        sudo sed -Ei "s/^(#)?(log_filters=).*/\2'4:*object* 4:*json* 4:*event* 4:*rpc* 4:daemon.remote 4:util.threadjob 4:*access* 1:*'/" /etc/libvirt/${daemon}.conf;
> +        sudo systemctl --quiet stop ${daemon}.service;
> +        sudo systemctl restart ${daemon}.socket;
> +      done
> +    - sudo virsh net-start default &>/dev/null || true;

What context is all this being run in ?

There's no docker image listed in the container, and I see the
'tags' referring to VMs.  Is this really installing stuff as
root in the VM runnerrs ?

Basically I'm not seeing where any of this work is cleaned up
to give a pristine environment for the next pipeline, or what
happens if two pipelines run concurrently ?

> +
> +  script:
> +    - mkdir logs
> +    - cd "$SCRATCH_DIR"
> +    - git clone --depth 1 https://gitlab.com/libvirt/libvirt-tck.git
> +    - cd libvirt-tck
> +    - sudo avocado --config avocado.config run --job-results-dir "$SCRATCH_DIR"/avocado
> +  after_script:
> +    - if test -e "$SCRATCH_DIR"/avocado;
> +      then
> +        sudo mv "$SCRATCH_DIR"/avocado/latest/test-results logs/avocado;
> +      fi
> +    - sudo mv /var/log/libvirt logs/libvirt
> +    - sudo chown -R $(whoami):$(whoami) logs
> +  variables:
> +    SCRATCH_DIR: "/tmp/scratch"
> +  artifacts:
> +    name: logs
> +    paths:
> +      - logs
> +    when: on_failure
> +
> +
> +libvirt-perl-bindings:
> +  stage: bindings
> +  trigger:
> +    project: eskultety/libvirt-perl

I assume pointing to this personal repo is a temp hack for some fix
that's not merged into main libvirt-perl ?

> +    branch: multi-project-ci
> +    strategy: depend
> +
> +
> +centos-stream-8-tests:
> +  extends: .tests
> +  needs:
> +    - libvirt-perl-bindings
> +    - pipeline: $PARENT_PIPELINE_ID
> +      job: x86_64-centos-stream-8
> +    - project: eskultety/libvirt-perl
> +      job: x86_64-centos-stream-8
> +      ref: multi-project-ci
> +      artifacts: true
> +  variables:
> +    DISTRO: centos-stream-8
> +  tags:
> +    - centos-stream-vm
> +
> +centos-stream-9-tests:
> +  extends: .tests
> +  needs:
> +    - libvirt-perl-bindings
> +    - pipeline: $PARENT_PIPELINE_ID
> +      job: x86_64-centos-stream-9
> +    - project: eskultety/libvirt-perl
> +      job: x86_64-centos-stream-9
> +      ref: multi-project-ci
> +      artifacts: true
> +  variables:
> +    DISTRO: centos-stream-9
> +  tags:
> +    - centos-stream-vm
> +
> +fedora-34-tests:
> +  extends: .tests
> +  needs:
> +    - libvirt-perl-bindings
> +    - pipeline: $PARENT_PIPELINE_ID
> +      job: x86_64-fedora-34
> +    - project: eskultety/libvirt-perl
> +      job: x86_64-fedora-34
> +      ref: multi-project-ci
> +      artifacts: true
> +  variables:
> +    DISTRO: fedora-34
> +  tags:
> +    - fedora-vm
> +
> +fedora-35-tests:
> +  extends: .tests
> +  needs:
> +    - libvirt-perl-bindings
> +    - pipeline: $PARENT_PIPELINE_ID
> +      job: x86_64-fedora-35
> +    - project: eskultety/libvirt-perl
> +      job: x86_64-fedora-35
> +      ref: multi-project-ci
> +      artifacts: true
> +  variables:
> +    DISTRO: fedora-35
> +  tags:
> +    - fedora-vm
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 4bcaf22ce2..453472c8be 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -4,6 +4,7 @@ variables:
>  stages:
>    - containers
>    - builds
> +  - test
>    - sanity_checks
>  
>  .script_variables: &script_variables |
> @@ -128,3 +129,16 @@ coverity:
>      - curl https://scan.coverity.com/builds?project=$COVERITY_SCAN_PROJECT_NAME --form token=$COVERITY_SCAN_TOKEN --form email=$GITLAB_USER_EMAIL --form file=@cov-int.tar.gz --form version="$(git describe --tags)" --form description="$(git describe --tags) / $CI_COMMIT_TITLE / $CI_COMMIT_REF_NAME:$CI_PIPELINE_ID"
>    rules:
>      - if: "$CI_PIPELINE_SOURCE == 'schedule' && $COVERITY_SCAN_PROJECT_NAME && $COVERITY_SCAN_TOKEN"
> +
> +integration:
> +  stage: test
> +  needs:
> +    - x86_64-centos-stream-8
> +    - x86_64-centos-stream-9
> +    - x86_64-fedora-34
> +    - x86_64-fedora-35
> +  trigger:
> +    include: .gitlab-ci-integration.yml
> +    strategy: depend
> +  variables:
> +    PARENT_PIPELINE_ID: $CI_PIPELINE_ID

I've not really thought about the implications, so I'm curious what's
the rationale for using a separate pipeline in this way, as opposed to
have x86_64-fedora-35-integration, x86_64-centos-stream-9-integration,
etc jobs in the existing pipeline ?

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|




[Index of Archives]     [Virt Tools]     [Libvirt Users]     [Lib OS Info]     [Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Big List of Linux Books]     [Yosemite News]     [KDE Users]     [Fedora Tools]

  Powered by Linux