depcheck test (was Re: measuring success)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Sat, 2010-07-03 at 12:27 +0200, Michael Schwendt wrote:

> [About automating this during the push process, it is possible to have
> a depchecker simulate a --skip-broken and exclude packages which break
> dependencies. There are different strategies. However, the procedure
> must be backed up by strong policies, or else we will see broken deps
> whenever somebody skips the automated depchecking to push "something
> important".]

This is approximately what the (in-progress) 'depcheck' test does. And
the proposed policy backing it is fairly strong: no package that fails
depcheck will be eligible for push to a live repo without rel-eng
override.

I've been meaning to write a blog post detailing the depcheck algorithm,
because getting the test right turns out to be *really* tricky. There
have been a lot of meandering discussions about how it *should* be done,
but it's complex enough that nobody's actually managed to sit down and
*do* it before now.

I'll attempt to give a brief summary here. First you need to understand
that there are three states for a package that has been built with the
hope of being pushed as an update:
* 'candidate': freshly-built packages intended for updates
* 'proposed': bodhi request filed, but not pushed live
* 'live': packages that have been pushed live by rel-eng

So. What we want to do in AutoQA is trigger a test to check dependencies
*before* the package goes live. So we trigger our test when the request
is filed (this is the 'post-bodhi-update' hook in AutoQA, which was
merged a week or two back).

It should be obvious that the goal of the test is to examine the
proposed updates and hold back the ones whose dependencies are
inconsistent with the packages in the live repos[1]. To phrase it
slightly differently: we want to look at the set of 'proposed' updates
and pass the packages whose dependencies *are* consistent with the live
repos.

As Michael correctly suggests, this is almost exactly the same as
running yum --skip-broken and seeing which packages can be installed. In
fact, we use the exact same depsolving code to determine this[2].

The first automated runs of this test should start this week - but it
may take a week or two of testing to ensure depcheck is running as
expected.

Once we're satisfied that depcheck does the right thing, we will
probably set it up to start adding automatic +1 karma from 'autoqa' when
updates pass the automated test suite (depcheck and possibly other tests
- rpmlint, rpmguard, etc.). We haven't yet decided if the autokarma will
apply to all packages or just critpath packages; it may apply
everywhere, which will make it a bit easier to get to +3 karma for
automated updates.

Once that's working we plan to add code to bodhi to show rel-eng which
updates have passed depcheck, so only those packages which won't break
the repos will get pushed. And then life will be lots nicer.

I know that's kind of a long explanation, but it's kind of a complicated
problem. Please feel free to examine the depcheck code and ask any
questions you like. If you have any "does it handle situation X"
questions, try to construct an actual test case (e.g. create some
trivial RPMs to illustrate the situation) and we'll put that into the
depcheck test suite.

If there are any other questions, feel free to ask.

-w

[1] Note that the test needs to check *all* the proposed updates, not
just the new individual update that is triggering the test. Otherwise
anything that got rejected would have to be resubmitted to re-trigger
the test (which would be annoying) and mutually dependent updates would
fail forever (which would be broken).

[2]
http://git.fedorahosted.org/git/?p=autoqa.git;a=blob;f=tests/depcheck/depcheck

-- 
devel mailing list
devel@xxxxxxxxxxxxxxxxxxxxxxx
https://admin.fedoraproject.org/mailman/listinfo/devel


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Fedora Announce]     [Fedora Kernel]     [Fedora Testing]     [Fedora Formulas]     [Fedora PHP Devel]     [Kernel Development]     [Fedora Legacy]     [Fedora Maintainers]     [Fedora Desktop]     [PAM]     [Red Hat Development]     [Gimp]     [Yosemite News]
  Powered by Linux