Hi, Petr,
On Fri, May 22, 2020 at 9:29 AM Petr Pisar <ppisar@xxxxxxxxxx> wrote:
>
> On Thu, May 21, 2020 at 07:29:35PM +0200, Aleksandra Fedorova wrote:
> > On Thu, May 21, 2020 at 7:04 PM Richard Shaw <hobbes1069@xxxxxxxxx> wrote:
> > >
> > > On Thu, May 21, 2020 at 11:56 AM Aleksandra Fedorova <alpha@xxxxxxxxxxxx> wrote:
> > >>
> > >> To run the command above as the integration test you need to put
> > >> tests/tests.yml file in dist-git repo with the following content:
> > >>
> > >> - hosts: localhost
> > >> roles:
> > >> - role: standard-test-basic
> > >> tags:
> > >> - classic
> > >> tests:
> > >> - simple:
> > >> dir: .
> > >> run: "FreeCAD -t 0"
> > >>
> > >> Note how here you will be using the install FreeCAD binary, not the local one.
> > >
> > >
> > > Is this only run on "real" builds, or is it possible to run locally or for scratch builds?
> >
> > We run it for pull-requests and for "real builds.
> >
> > It runs in the following way:
> >
> > 1) we take latest Fedora Rawhide qcow image,
> > 2) we run virtual machine via qemu-kvm from it,
> > 3) inside the vm we install the package,
> > 4) we run the command defined by "run" key in that YAML file.
> >
> > And it is all orchestrated by Ansible.
> >
> > It is possible to reproduce the test locally with all the CI wrappers
> > around it, but I wouldn't recommend it. To debug the test I would just
> > run the VM with the same Rawhide image [1], install the package and
> > run the test command manually.
>
> I'd like to point out that there is a difference in the enviroments between
> these two approaches.
>
> While the manual procedure only installs a binary package (and its
> dependencies), the OSCI procedure installs all binary packages produced by the
> source package (and their depenencies).
You are right here. Jenkins pipeline tries to install all packages
from the koji build before running the test.
And this approach has already hit us many times: there are packages
where binary subpackages conflict with each other (as in curl), or
where different tests require different subpackages to be installed.
There is also a case for reverse dependency check, where we would like
to run test suite from a package A to test the package B, where the
koji build we are installing is different from what is expected by the
test.
In Fedora CI SIG we are discussing if we should just disable this
functionality and make it mandatory to specify all packages explicitly
in the tests.yml under "required_packages" tag, as in [1]
The downside is:
you would need to maintain list of required packages in the tests.yml
We haven't brought this topic to a wider audience yet, but we should.
> In case of freecad, it does not matter, because the freecad component has only
> one binary package. But in general it's not true.
>
> And here I'd like to ask Aleksandra whether the same applies to modules or
> not. In general case, a module build produces two binary modules - a non-devel
> module and a devel module. Does OSCI install components from both of them? And does
> it install the components explicitly, or does it rather install a default
> profile of the module ("dnf module install" command)?
We don't have test for modules yet in Fedora CI. So I would say the
question here is:
how do you like it to work?
We can do "dnf module install" but then the same issues would appear:
if you need additional flexibility to test parts of the module, you
would need to revert this default behavior somehow.
Or we can do nothing and only provide environment with repositories
with the updates. So that configuration of the environment is done in
the test metadata.
[1] https://docs.fedoraproject.org/en-US/ci/how-to-add-dist-git-test/#_tests_in_a_subpackage
--
Aleksandra Fedorova
bookwar
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
