On Thu, Oct 13, 2022 at 09:06:53AM +0100, Daniel P. Berrangé wrote:
> On Thu, Oct 13, 2022 at 09:43:50AM +0200, Erik Skultety wrote:
> > On Mon, Oct 10, 2022 at 11:09:51AM +0100, Daniel P. Berrangé wrote:
> > > On Mon, Oct 10, 2022 at 11:46:43AM +0200, Peter Krempa wrote:
> > > > After addition of the new libvirt-client-qemu sub-package which is using
> > > > python bindings (thus creating a circular dependency between the libvirt
> > > > and libvirt-python projects) the integration jobs fail with:
> > > >
> > > > Error:
> > > > Problem: conflicting requests
> > > > - nothing provides python3-libvirt >= 8.9.0-1.el9 needed by libvirt-client-qemu-8.9.0-1.el9.x86_64
> > >
> > > Hmm, I wonder, should we really be trying to install *all* RPMs ?
> > >
> > > IIUC, we're only running integration tests for QEMU, so it ought
> > > to be sufficient to do only have the QEMU driver and virsh client
> > > and Perl APIs. eg can we change ci/integrate-templates.yml to
> > > do
> > >
> > > createrepo libvirt-rpms
> > > createrepo libvirt-perl
> > > dnf install libvirt-daemon-qemu libvirt-client perl-Sys-Virt
> >
> > Sorry for the delay, I was actually integrating ^this suggestion and GitLab is,
> > well, slow to test these changes with. Anyhow, I vote for Peter's original
> > patch as it's the simplest and cleanest way to fix the problem, why? Here's
> > what I ended up with on my branch:
> >
> > diff --git a/ci/integration-template.yml b/ci/integration-template.yml
> > index d4799b2ca8..9c58a1b71c 100644
> > --- a/ci/integration-template.yml
> > +++ b/ci/integration-template.yml
> > @@ -25,7 +25,31 @@
> >
> >
> > .install-deps: &install-deps
> > - - sudo dnf install -y libvirt-rpms/* libvirt-perl-rpms/*
> > + - sudo dnf install -y createrepo
> > + - createrepo libvirt-rpms
> > + - createrepo libvirt-perl-rpms
> > + - |-
> > + for repo in libvirt-rpms libvirt-perl-rpms; do
> > + cat << EOF >> "$SCRATCH_DIR/ci.repo"
> > + [CI-$repo]
> > + name=ci-$repo
> > + baseurl=file://$(pwd)/$repo
> > + enabled=1
> > + gpgcheck=0
> > + EOF
> > + done
> > + - sudo mv "$SCRATCH_DIR/ci.repo" /etc/yum.repos.d/
> > + - sudo dnf install -y
> > + libvirt-daemon-config-nwfilter
> > + libvirt-daemon-config-network
> > + libvirt-daemon-driver-interface
> > + libvirt-daemon-driver-network
> > + libvirt-daemon-driver-nodedev
> > + libvirt-daemon-driver-nwfilter
> > + libvirt-daemon-driver-qemu
> > + libvirt-daemon-driver-secret
> > + libvirt-daemon-driver-storage
> > + perl-Sys-Virt
> > - sudo pip3 install --prefix=/usr avocado-framework
>
> FYI, can be simplified to:
>
> - sudo dnf install -y createrepo
> - createrepo libvirt-rpms
> - createrepo libvirt-perl-rpms
> - dnf config-manager --add-repo libvirt-rpms
> - dnf config-manager --add-repo libvirt-perl-rpms
Oh, I didn't know DNF could do this^^^, that's neat. I still prefer the
simplicity of Peter's patch though.
> - sudo dnf install -y
> libvirt-daemon-config-nwfilter
> libvirt-daemon-config-network
> libvirt-daemon-qemu
> perl-Sys-Virt
>
> > Not only does it require more steps, is ugly because of the heredoc combined
> > with YAML's explicit line break preservation (|-), libvirt-daemon-driver-qemu
> > also doesn't pull all the needed bits as it was suggested, e.g.
> > libvirt-daemon-config-nwfilter which is needed by the tests. Although the tests
> > don't need virtinterfaced, I still had to include it because the logic which
> > restarts all the daemons would have to be adjusted as well; and then one day we
> > add a test for virtinterfaced we'd have to add it back, so IMO installing and
> > restarting all the daemons is the cleanest and most transparent way of setting
> > up the CI env even though we're not making use of all of it now (and so I ended
> > up naming all the secondary drivers explicitly).
>
> NB, I suggested libvirt-daemon-qemu, not libvirt-daemon-driver-qemu,
> because the former pulls in all the secondary driver packages, so
> would only miss the config packages.
We don't build libvirt-daemon-qemu on CS9, I presume that's a bug.
>
> > The other huge inconvenience with the proposed approach compared to Peter's is
> > debugging the failures. What I currently do with any integration CI failures is
> > that I connect to the runner machine and spin up an instance of one of the
> > images from which gitlab provisions the environment, download the RPM artifacts
> > from libvirt and libvirt-perl, install and then debug - again, why? Because
> > unfortunately trying to do this locally is still less convenient even though
> > we've done plenty of work in lcitool to improve the status quo. So, back to
> > your suggestion - I'd have to create the repositories manually, install a few
> > explicit packages and then test; sorry, that maybe fine for the CI, but doesn't
> > beat the following in any way:
> >
> > wget <libvirt-rpms-uri> -O libvirt_rpms.zip
> > wget <libvirt-perl-rpms-uri> -O libvirt_perl_rpms.zip
> > wget <libvirt-python-rpms-uri> -O libvirt_python_rpms.zip
> > unzip <yada yada>
> >
> > dnf install libvirt_rpms/* libvirt_perl_rpms/* libvirt_python_rpms/*
>
> That's fine as an approach for reproducing locally. Does it matter
> that you're installing more packages that the CI system would have ?
> I guess it could affect the results, but fairly slim liklihood IMHO.
No, the point I was trying to make is that for ^this use case we still need
Peters libvirt-python MR (which I merged in the meantime btw). The other
argument I have is that we shouldn't deviate in the CI job transcript from what
you would done naturally locally in a manual manner - sure, you can always
copy-paste, but copy pasting
dnf install libvirt_rpms/* libvirt_perl_rpms/* libvirt_python_rpms/*
will always be faster than copying
sudo dnf install -y createrepo
createrepo libvirt-rpms
createrepo libvirt-perl-rpms
createrepo libvirt-python-rpms
dnf config-manager --add-repo libvirt-rpms
dnf config-manager --add-repo libvirt-perl-rpms
dnf config-manager --add-repo libvirt-python-rpms
sudo dnf install -y
libvirt-daemon-config-nwfilter
libvirt-daemon-config-network
libvirt-daemon-qemu
perl-Sys-Virt
Erik
