On Mon, 2020-11-30 at 00:24 +0100, Dan Čermák wrote: > Neal Gompa <ngompa13@xxxxxxxxx> writes: > > > Richard Brown from openSUSE also informed me that it's technically > > possible to do some level of audio subsystem QA using OpenQA, but I'm > > not sure how to do it. Perhaps that's another avenue we can pursue to > > improve integration testing coverage? > > While it is technically possible to do audio testing in openQA, it is > not really used a lot. The main reason is that the current > implementation is rather brittle (albeit pretty clever): it converts the > audio signal to an image (afaik it plots the intensity) and does its > usual needle comparison. Unfortunately this does not work that well in > practice and the one of the few audio tests that openSUSE has in openQA > is a constant source of trouble, while catching very few bugs [1]. > > But even if openQA had a very reliable way to test audio, I'm afraid it > wouldn't really help us out too much here. openQA is realistically only > going to be able to cover your basic use cases, but it's not going to be > able to test all the special audio setups that all the people who are > into audio have at home (as that would require their hardware & > software). So openQA would only test what the PipeWire devs are probably > testing already anyway and it would not provide a whole lot of > additional interesting test cases. > > Now, this does not mean that we shouldn't test audio in Fedora's openQA > (we don't atm), as that is still a valuable regression test. But we > won't be able to cover a lot of the more interesting use cases where I > would expect [2] that most of current the bugs are. Those will only get > found by running this on real world hardware by folks that have their > working setups. Yeah, I looked into it before and this is basically why I didn't bother implementing it; it doesn't buy us much more than we're getting naturally from human testing anyway. -- Adam Williamson Fedora QA IRC: adamw | Twitter: adamw_ha https://www.happyassassin.net _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
