On Thu, 22 Aug 2013 11:03:52 -0400
Matthew Miller <mattdm@xxxxxxxxxxxxxxxxx> wrote:
> Based on discussion at Flock, on the devel mailing list, and in the
> FESCo meeting, we are looking for feedback on the idea of a longer
> release cycle for Fedora 21 -- not (right now at least) the bigger
> question of the 6-month cycle overall, but just, right now, slowing
> down for a release to get some things in order.
>
> Specifically, both Release Engineering and QA have clear needs (and
> even plans for) greater automatiion, but are also incredibly busy
> simply doing the things they need to do _now_ to get the release out
> the door.
>
> So, FESCo would like to see some specifics, like "If we had one week
> with nothing else to worry about, we could have automated generation
> and upload of cloud images" (to pick an example I personally care
> about). Or "with six months of overall delay, we could have
> continuous integration testing of a key subset of rawhide". Or "we
> could spend a couple of weeks and automate the new package and review
> workflow".
For QA tools/automation, a week isn't going to give us much - we already
have a couple of weeks between releases and we're more blocked by
projects that will take longer than a week.
I've been meaning to sit down and come up with a more detailed
list/plan for qa development (automation, other tools) but that hasn't
happened yet. At the end of this email, I made a list of the things that
I've been talking about with various folks in the order of both how
important I think they are and how much the project would benefit from
a more extended break between releases. Exactly how much of this we
could do with more time depends on both how much time we're talking
about and who all would be involved.
Tim
Taskbot [1]:
This will become the foundation for future automation work and at
the moment is at least somewhat blocking our other automated testing
initiatives from moving forward. This would (eventually, not all of
this would be part of the first deliverable) give us:
- easier for new people to get up to speed and help
creating/maintaining checks/tasks
- more flexibility in the types of checks/tasks that could be
automated
- better triggering (run X check for builds of Y package, run Z at a
certain time etc.)
- better reporting
- automated analysis of logs for oddities or to answer questions
like "how long have we been seeing this error in syslogs"
[1] http://tirfa.com/tag/taskbot.html
Automated Install Testing:
Many of our current validation test cases [2] are very straight
forward and could be automated to free up human testers to do other
testing that isn't (easily) automate-able.
We have a start on this with infinity [3] but there is still some
development work, a lot of integration work to do and test cases to
write before any of this is usable.
[2] https://fedoraproject.org/wiki/QA:Fedora_20_Install_Results_Template
[3] https://github.com/garretraziel/infinity
Smoke image build automation:
This has been started [4] as part of GSoC 2012 but is still a little
shy of being usable in production. The idea would be to build images
as soon as new packages (anaconda, maybe others) so that a set of
automated install smoke tests could be kicked off. This could involve
working with releng on something to do the composes - I'm less
interested in who does the work than I am in being able to get images
to test on demand.
[4] https://fedorahosted.org/fedora-build-service/
Test Case and Results Management:
We want to replace our current wiki-based system of test cases and
results matrices. I'm not aware of any existing system that would
fit our needs and I think we're going to end up rolling our own
unless something new shows up.
Update/Build Gating:
There's been talk about gating updates based on automated test
results for a while but nothing's finished yet. A lot of this is
integration with bodhi/koji but there are still some bits that
haven't been implemented (test result manual override is the first
thing that comes to mind)
Better Automated Checks:
Rewriting depcheck to be more useable, abi breakage checks, running
gnome's new test suite or anything else that people can come up with.
This can happen just as easily in parallel with releases once we have
the infrastructure in place to run them, though and doesn't really
require an extended break between releases.
> What Infrastructure projects would be helped by this? Web and design
> team, would slowing down the release focus allow time to work on, oh,
> say, getting the Wiki beautiful (or does it not matter)? What else?
>
> As we look at Fedora.next ideas and possibly decide to start
> implementation in the F21 timeframe, we will likely find _new_ things
> that take specific work. Let's not worry about that right now. What
> things we do _now_ could be improved with the investment of some
> effort?
>
>
Attachment:
signature.asc
Description: PGP signature
-- devel mailing list devel@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
