On Thu, Oct 13, 2022 at 09:06:53AM +0100, Daniel P. Berrangé wrote: > On Thu, Oct 13, 2022 at 09:43:50AM +0200, Erik Skultety wrote: > > On Mon, Oct 10, 2022 at 11:09:51AM +0100, Daniel P. Berrangé wrote: > > > On Mon, Oct 10, 2022 at 11:46:43AM +0200, Peter Krempa wrote: > > > > After addition of the new libvirt-client-qemu sub-package which is using > > > > python bindings (thus creating a circular dependency between the libvirt > > > > and libvirt-python projects) the integration jobs fail with: > > > > > > > > Error: > > > > Problem: conflicting requests > > > > - nothing provides python3-libvirt >= 8.9.0-1.el9 needed by libvirt-client-qemu-8.9.0-1.el9.x86_64 > > > > > > Hmm, I wonder, should we really be trying to install *all* RPMs ? > > > > > > IIUC, we're only running integration tests for QEMU, so it ought > > > to be sufficient to do only have the QEMU driver and virsh client > > > and Perl APIs. eg can we change ci/integrate-templates.yml to > > > do > > > > > > createrepo libvirt-rpms > > > createrepo libvirt-perl > > > dnf install libvirt-daemon-qemu libvirt-client perl-Sys-Virt > > > > Sorry for the delay, I was actually integrating ^this suggestion and GitLab is, > > well, slow to test these changes with. Anyhow, I vote for Peter's original > > patch as it's the simplest and cleanest way to fix the problem, why? Here's > > what I ended up with on my branch: > > > > diff --git a/ci/integration-template.yml b/ci/integration-template.yml > > index d4799b2ca8..9c58a1b71c 100644 > > --- a/ci/integration-template.yml > > +++ b/ci/integration-template.yml > > @@ -25,7 +25,31 @@ > > > > > > .install-deps: &install-deps > > - - sudo dnf install -y libvirt-rpms/* libvirt-perl-rpms/* > > + - sudo dnf install -y createrepo > > + - createrepo libvirt-rpms > > + - createrepo libvirt-perl-rpms > > + - |- > > + for repo in libvirt-rpms libvirt-perl-rpms; do > > + cat << EOF >> "$SCRATCH_DIR/ci.repo" > > + [CI-$repo] > > + name=ci-$repo > > + baseurl=file://$(pwd)/$repo > > + enabled=1 > > + gpgcheck=0 > > + EOF > > + done > > + - sudo mv "$SCRATCH_DIR/ci.repo" /etc/yum.repos.d/ > > + - sudo dnf install -y > > + libvirt-daemon-config-nwfilter > > + libvirt-daemon-config-network > > + libvirt-daemon-driver-interface > > + libvirt-daemon-driver-network > > + libvirt-daemon-driver-nodedev > > + libvirt-daemon-driver-nwfilter > > + libvirt-daemon-driver-qemu > > + libvirt-daemon-driver-secret > > + libvirt-daemon-driver-storage > > + perl-Sys-Virt > > - sudo pip3 install --prefix=/usr avocado-framework > > FYI, can be simplified to: > > - sudo dnf install -y createrepo > - createrepo libvirt-rpms > - createrepo libvirt-perl-rpms > - dnf config-manager --add-repo libvirt-rpms > - dnf config-manager --add-repo libvirt-perl-rpms Oh, I didn't know DNF could do this^^^, that's neat. I still prefer the simplicity of Peter's patch though. > - sudo dnf install -y > libvirt-daemon-config-nwfilter > libvirt-daemon-config-network > libvirt-daemon-qemu > perl-Sys-Virt > > > Not only does it require more steps, is ugly because of the heredoc combined > > with YAML's explicit line break preservation (|-), libvirt-daemon-driver-qemu > > also doesn't pull all the needed bits as it was suggested, e.g. > > libvirt-daemon-config-nwfilter which is needed by the tests. Although the tests > > don't need virtinterfaced, I still had to include it because the logic which > > restarts all the daemons would have to be adjusted as well; and then one day we > > add a test for virtinterfaced we'd have to add it back, so IMO installing and > > restarting all the daemons is the cleanest and most transparent way of setting > > up the CI env even though we're not making use of all of it now (and so I ended > > up naming all the secondary drivers explicitly). > > NB, I suggested libvirt-daemon-qemu, not libvirt-daemon-driver-qemu, > because the former pulls in all the secondary driver packages, so > would only miss the config packages. We don't build libvirt-daemon-qemu on CS9, I presume that's a bug. > > > The other huge inconvenience with the proposed approach compared to Peter's is > > debugging the failures. What I currently do with any integration CI failures is > > that I connect to the runner machine and spin up an instance of one of the > > images from which gitlab provisions the environment, download the RPM artifacts > > from libvirt and libvirt-perl, install and then debug - again, why? Because > > unfortunately trying to do this locally is still less convenient even though > > we've done plenty of work in lcitool to improve the status quo. So, back to > > your suggestion - I'd have to create the repositories manually, install a few > > explicit packages and then test; sorry, that maybe fine for the CI, but doesn't > > beat the following in any way: > > > > wget <libvirt-rpms-uri> -O libvirt_rpms.zip > > wget <libvirt-perl-rpms-uri> -O libvirt_perl_rpms.zip > > wget <libvirt-python-rpms-uri> -O libvirt_python_rpms.zip > > unzip <yada yada> > > > > dnf install libvirt_rpms/* libvirt_perl_rpms/* libvirt_python_rpms/* > > That's fine as an approach for reproducing locally. Does it matter > that you're installing more packages that the CI system would have ? > I guess it could affect the results, but fairly slim liklihood IMHO. No, the point I was trying to make is that for ^this use case we still need Peters libvirt-python MR (which I merged in the meantime btw). The other argument I have is that we shouldn't deviate in the CI job transcript from what you would done naturally locally in a manual manner - sure, you can always copy-paste, but copy pasting dnf install libvirt_rpms/* libvirt_perl_rpms/* libvirt_python_rpms/* will always be faster than copying sudo dnf install -y createrepo createrepo libvirt-rpms createrepo libvirt-perl-rpms createrepo libvirt-python-rpms dnf config-manager --add-repo libvirt-rpms dnf config-manager --add-repo libvirt-perl-rpms dnf config-manager --add-repo libvirt-python-rpms sudo dnf install -y libvirt-daemon-config-nwfilter libvirt-daemon-config-network libvirt-daemon-qemu perl-Sys-Virt Erik