On Fri, Feb 12, 2021 at 2:28 PM Lukas Ruzicka <lruzicka@xxxxxxxxxx> wrote:
List of test cases to test that:
- PipeWire is used by default (https://fedoraproject.org/wiki/User:Lruzicka/QA:Testcase_pipewire_default)
s/bare machines/bare-metal machines/
(Also in (all) other testcases)
- PipeWire CLI works (https://fedoraproject.org/wiki/User:Lruzicka/QA:Testcase_pipewire_cli)
"Have an audio file ready (wav)."
It would be nice either to link to some freely available sample, or specify a path where such a file is present on default desktop installations (there's usually something in /usr/share/sounds/, for example). Otherwise each tester needs to waste time searching for a wav file individually.
"Play a sound file over the default target and check that it is correctly played over the default sound device.
$ pw-cat -p <audiofile>"
$ pw-cat -p <audiofile>"
Is this supposed to be the wav file, or any audio file (e.g. ogg)? A clarification would be good, e.g.:
"Play a wav sound file over the default target and check that it is correctly played over the default sound device.
$ pw-cat -p <audiofile.wav>"
$ pw-cat -p <audiofile.wav>"
- ALSA backend works (https://fedoraproject.org/wiki/User:Lruzicka/QA:Testcase_ALSA_backend)
"Log in as a common user."
A "regular user", if you want to make it consistent.
s/headphoned/headphones/
"Keep an audio file at hand (wav)."
Ditto as above.
The Expected Results section could be simplified, either merged into the steps, or e.g. "Every time you use `aplay` to play an audio file in each testing step, you can hear it correctly".
- Audio Selected Apps (https://fedoraproject.org/wiki/User:Lruzicka/QA:Testcase_audio_selected_apps)
- Pavucontrol works (https://fedoraproject.org/wiki/User:Lruzicka/QA:Testcase_pavucontrol_basic)
LGTM
- Pavucontrol mixing (https://fedoraproject.org/wiki/User:Lruzicka/QA:Testcase_pavucontrol_mixing)
"Connect your speakers (headphones) to all your computer’s sound devices."
This confused me until I read the whole testcase. It sounds like I should connect the same headphones to all sound outputs, which is of course impossible to do at the same time, so I thought I would re-plug it to each device as needed. Turns out you wanted to say "make sure each sound device has some speakers/headphones connected to it", right? It would be good to make this clearer.
- Pavucontrol recording (https://fedoraproject.org/wiki/User:Lruzicka/QA:Testcase_pavucontrol_recording)
LGTM
- Pavucontrol add and remove devices (https://fedoraproject.org/wiki/User:Lruzicka/QA:Testcase_pavucontrol_addremove)
"The computer must be equipped with a sound device with multiple ports (speakers and phones), such as laptops."
Perhaps you can also say "or desktops with an audio front panel (where plugging a jack into the front audio panel would disable the usual jack output on the computer's back)" or similar, so that people know laptops are not strictly needed.
"Plug in an HDMI monitor and check that the device appears in the Pavucontrol’s list of devices."
I'm a bit confused that Setup didn't speak about HDMI at all and suddenly in step 8 I need it. If it is a requirement, it should be in Setup. If it is just optional, it should be said so ("If you have a HDMI monitor, ...").
Also:
a) audio routing is available not just with HDMI, but also with DisplayPort (which might be even more common, for desktop users)
b) just a HDMI/DP monitor is not enough, I think. It has to either have its own speakers, or an output jack (which you can plug headphones/speakers into). If it doesn't have any audio output, I'm not sure whether it appears as an audio output device once connected (and even if it did, you wouldn't be able to verify it works).
- PipeWire handles Bluetooth devices (https://fedoraproject.org/wiki/User:Lruzicka/QA:Testcase_bluetooth_devices)
"The computer must be equipped with a Bluetooth adapter."
And also you need to have some BT device to test with. Those can be BT-enabled headphones or speakers. Is there anything else? Can I test it using a phone somehow? Just wondering.
- Jack applications (https://fedoraproject.org/wiki/User:Lruzicka/QA:Testcase_jack)
"Install the pipewire-jack-audio-connection-kit package. Using the --allowerasing option might be necessary for successful installation."
This might be confusing if one doesn't know the option and therefore doesn't realize you're talking about dnf.
- Jack MIDI routing (https://fedoraproject.org/wiki/User:Lruzicka/QA:Testcase_jack_midi)
"Install the pipewire-jack-audio-connection-kit package. Using the --allowerasing option might be necessary for successful installation."
Ditto.
Overall it looks very good. Note that some time ago I created a special category for "test day's test cases", and so let's put all of those in when moving them into the final location:
Since those testcases are related to the PipeWire testday, you can prefix them like that, e.g. "Testcase PipeWire ALSA backend" or "Testcase PipeWire Jack". Just a thought.
As discussed yesterday at our team meeting, adding one more testcase to test popular videoconferencing tools is probably a good idea.
Thanks for your work,
Kamil
_______________________________________________ test mailing list -- test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to test-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
