Re: Questions about the testing process

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Thu, 2021-06-17 at 11:49 -0300, Beatriz Michelson Reichert wrote:
>  Hi Adam,
> 
> Are release validation test results forwarded to the ResultsDB repository?

Yes. fedora_openqa has a fedora-messaging consumer which listens out
for "test complete" messages and submits results to resultsdb.

> If so, after this storage, if the compose passes the tests, the
> synchronization with the mirrors starts. Who is responsible for initiating
> the synchronization?

This is not actually how it works yet for automated nightly composes of
Rawhide and Branched (when Branched exists). In those cases, the tests
are currently 'advisory'. If the compose succeeds - i.e. if all 'fatal'
compose tasks complete - it gets synced out to become the 'current'
content of the release repository; the test results do not factor in.
The synchronization is part of release engineering's overall compose
process. I used to know where the actual script was, but either it
moved or I forgot! Mohan Boddu could tell you.

We (Fedora) have had a plan approximately forever to make the nightly
composes sync only if a given subset of the openQA tests pass, but it's
never been fully implemented. We (QA) did implement all the bits that
are our responsibility: the greenwave policy that was supposed to be
used for this exists -
https://pagure.io/fedora-infra/ansible/blob/main/f/roles/openshift-apps/greenwave/templates/fedora.yaml#_156
- and we even set up the daily "compose check reports" to log whether
the compose passed that check or not, so we could see how often it
passes and fails. But for whatever reason, things never progressed
beyond that point. The next step would be for releng to adjust the
compose process to actually wait after the compose is complete and only
proceed to the sync step if the greenwave policy is satisfied.

For the process of doing stable releases, this all happens manually in
a loop between releng and us, basically. We (usually I) request a
candidate compose via a Pagure ticket. releng manually triggers the
compose. It is synced from the initial output location
under https://kojipkgs.fedoraproject.org/compose/ to the stage location
- https://dl.fedoraproject.org/pub/alt/stage/ - by releng (not sure if
this is manual or automated); that location is mirrored but only
lightly.

The automated tests run automatically as soon as the compose is
complete, the 'relvalconsumer' message consumer will automatically
create a validation event in the wiki and announce it, and then humans
run the manual tests. A candidate compose would only be 'synced'
further than the stage location if it were actually released as the
Beta or Final release, which happens after we do a go/no-go meeting and
sign off on the release. That sync process is also handled manually, I
believe (using a script).

HTH!
-- 
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha
https://www.happyassassin.net


_______________________________________________
test mailing list -- test@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to test-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/test@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure




[Index of Archives]     [Fedora Desktop]     [Fedora SELinux]     [Photo Sharing]     [Yosemite Forum]     [KDE Users]

  Powered by Linux