Re: Future of Test Cases and their Management in Fedora QA

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Wed, 2022-06-22 at 08:57 -0600, Tim Flink wrote:


> ================================================================================
> 3. Continue Using WikiTCMS
> ================================================================================
> 
> Advantages:
>   - Doesn't require any changes to how we're doing things
>   - Is very flexible - can do pretty much whatever we want to without a ton of code
>   - Doesn't require much maintenance or custom code beyond wikitcms

Another advantage of the current system that I'd like to highlight is
that it allows us to synthesize manual and automated test results. That
is, in the current wiki results pages, you see the results from openQA
and human testers together.

I would definitely want any replacement for it to preserve this
advantage - i.e., I would want it to be able to track results from both
human testers and automated systems. Ideally, this would be achieved by
resultsdb integration - it's a bit difficult to imagine technically how
this could work, but it would be nice for results from both human
testers and automated systems to be in resultsdb, and for any human-
readable representation of the validation state to pull both sets of
results from resultsdb.

wikitcms gives us the integration of the two types of result, but it
does not currently forward results from human testers to resultsdb. So
we only have the results from automated systems in resultsdb. openQA
reports its results to *both* resultsdb and the wiki, so the wiki is
the only canonical store of *both* human and bot results.

I've toyed in the past with the idea of having `relval report-results`
also submit the result to resultsdb, but it requires solving
authentication and we still wouldn't capture human test results
submitted by editing the wiki directly. Another option would be a
wikitcms-based bot which monitored result pages for edits and forwarded
any 'new results' to resultsdb...

> 
> Disadvantages:
>   - More difficult to analyze data contained in the wiki
>   - Only one person can report results in a given matrix at a time
>   - QA is the only group still using the Fedora wiki
> 
> This kinda started a while back when CPE started asking about whether
> we still needed the wiki or not. The current Fedora wiki is a source
> of frustration for them due to the amount of spam and bot issues that
> they have to deal with. Seeing as how QA is one of the only (if not
> the only) groups left using the wiki, it's worth asking whether we
> really still need it or not. It does sound like most of CPE's
> concerns could be satisfied with a new wiki instance which could be
> locked down to only allow pages with a certain format but I don't
> believe that conversation has even been started with them.

If they do get to the point where they really, really want to turn the
wiki off, I'm willing to try and handle this (setting up a reduced wiki
instance specifically for this purpose, with restrictions to allowable
page names and so on).
-- 
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha
https://www.happyassassin.net

_______________________________________________
test mailing list -- test@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to test-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/test@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure




[Index of Archives]     [Fedora Desktop]     [Fedora SELinux]     [Photo Sharing]     [Yosemite Forum]     [KDE Users]

  Powered by Linux