On Thu, 2017-11-09 at 20:37 -0500, Sumantro Mukherjee wrote: > Hey All, > > Now that we have the F27 Modular Server Beta, It will be great to have the test cases. > I've started writing them, I would like to have the feedback and thoughts on them. > I have mostly used dnf module --help as a base to derive the these. If something is > missing, I would love to draft them too, just let me know. > > [1] Listing all modules: https://fedoraproject.org/wiki/User:Sumantrom/Draft/FMS_module_list > [2] Enabling & Disabling modules: https://fedoraproject.org/wiki/User:Sumantrom/Draft/FMS_enable_disable_module > [3] Install & Update : https://fedoraproject.org/wiki/User:Sumantrom/Draft/dnf_install_remove_update_module > [4] Module Info : https://fedoraproject.org/wiki/User:Sumantrom/Draft/dnf_module_info Along with the suggestions from others, I'd say: 1) Make the setup steps more generic. It is a bad idea to talk about 'Fedora 27' and 'modular server' and even worse to include download links. Think about the philosophy behind the whole release validation design in the wiki: the test cases are made as generic ("vague") as possible. All the context about running the test for a specific release, or image, or whatever, is provided at a different level: a validation event page which specifies the compose it's for and provides download links, for instance, or a Test Day, which similarly specifies what particular image etc. is being tested at that Test Day. We keep that information in the validation event page or the Test Day page, not in the test case, so we don't have to continually update test case pages and duplicate them when we want to run the same test in a different context. Take a look at the steps in existing post-install validation test cases, for e.g. https://fedoraproject.org/wiki/QA:Testcase_base_update_cli . The text there is a pretty good guide: "Clean boot the Fedora you wish to test: this could be a system installed from a particular snapshot, pre-release, or release, or a live image. It should be an image for which updates will be available (or you can downgrade a package after installation)." I'd say to use something like that, but just specify that it must be a *modular* installation. 2) Similarly, the test case names should *not* include 'FMS' or any other reference to modular server specifically; in future, more things than just Server are going to use the Modularity stuff. I'd suggest consistent names something like dnf_module_(something). Perhaps: dnf_module_list dnf_module_enable_disable dnf_module_install_remove_update dnf_module_info 2) Remove the 'optional' sections - what you've left in is just sample text from the template, don't leave it in if there aren't actually any optional steps. (Also you seem to have some sort of template syntax error in User:Sumantrom/Draft/dnf_module_info ). 3) FMS_enable_disable_module should probably include some kind of verification as to whether the enablement/disablement actually worked. 4) I'm not sure I like the capitalization of Module, Stream, Install Profile etc in dnf_install_remove_update_module . I'd probably put them in italics instead - ''module'', ''stream'', etc. 5) Similarly to 3), the install_remove_update test should include verification that the installation / removal / update was actually performed. 6) It might be worth separating install/remove/update into two tests, one for update, one for install/remove. Thanks for wokring on this! -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net _______________________________________________ test mailing list -- test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to test-leave@xxxxxxxxxxxxxxxxxxxxxxx
