On Mon, 2009-06-29 at 15:27 -0400, Matthias Clasen wrote: > If you have ideas for > other areas that could benefit from this kind of attention, please let > us know. I can think of a number of different cross-component tests... * Audio Applications - Does the sound hardware work out of the box? Do all applications that produce audio output work out of the box? (For example, I had trouble in Fedora 9 with Rosegarden, which needs the Jack audio server set up in addition to PulseAudio, so it doesn't work out of the box.) Does it work if you try to use the application while another application is already producing audio output? Does volume control work? See: https://fedoraproject.org/wiki/Bugs/Common#PulseAudio-based_volume_control_cannot_adjust_volume_satisfactorily * Network Audio - Is it easy to play music on one computer and listen to it from the speakers on a different computer on the same subnet? * File Formats - For each given file type in the known universe (.txt, .doc, .jpg, .ogg, .mp3, .iso, etc.) are they handled properly by each application you would expect to do so? Files not handled internally should be passed off to the right application by all web browsers, IRC/instant message clients, Nautilus, etc. All of these applications should use the same universal list of handlers. If the necessary handler is not installed, one of these new features should be triggered: http://fedoraproject.org/wiki/Releases/FeatureCodecBuddy https://fedoraproject.org/wiki/Features/AutoFontsAndMimeInstaller * Help - Does activating the help system in each application work? Does it actually provide helpful information? Is documentation for non-network-oriented applications accessible even without a network connection? * Application Capabilities - Are applications honest about what file types they can open (e.g. when you select File -> Open from the menu, is there a filter which shows you only openable files)? If a given application can handle a given type of file, does that show up in the list of handlers in other applications (like a web browser)? If you open a file from within an application which it cannot handle, does something sensible happen (as opposed to crashing, getting gibberish, etc.)? Do all applications handle non-ASCII filenames properly? * Cut and Paste - Does it work smoothly everywhere, whether using mouse or keyboard shortcuts? Even in the To: and Cc: fields when composing an email? Even in the URL bar? In gnome-terminal? Does complex data (e.g. formatted text, graphics, multi-line text in single-line input boxes) get transferred properly? Most importantly, does cut-and-paste handle non-ASCII text encodings properly (without corrupting characters)? * International Content - Inside documents and in their own menus and GUI, applications should generally be able to render glyphs in any language or Unicode symbol set. If a suitable font is not installed, this should kick in: https://fedoraproject.org/wiki/Features/AutoFontsAndMimeInstaller In some cases, suitable free fonts do not yet exist, but these cases are generally quite exotic. A complex example: https://bugzilla.redhat.com/show_bug.cgi?id=455510 (Undisplayable glyphs on Wikipedia) * Menus - When someone unfamiliar with the layout thinks, "I want to do ___", is it easy to navigate to that function in the Gnome menus? If you install Everything, are all programs filed in the right place? Do all programs have a suitable menu icon? Is menu placement consistent with Add/Remove groups (e.g. Sound & Video vs Multimedia)? * Small Screens - Is everything still usable even when the screen resolution is 800x600 or 648x480? Most importantly, no functionality should be lost. One good test is to open each application and go to Edit -> Preferences; applications with too many preferences will often produce a window that is not scrollable even though it is too big to fit on the screen. Less important but good to test, is whether merely ugly or annoying things happen (like an application starting up with a window that is larger than the screen) which could be handled more gracefully. Is there a Fedora desktop configuration suitable for PDAs or other small mobile devices? * Press All the Buttons - Do all the keys you never use (Print Screen, Pause/Break, Windows, Menu, Play/Stop/Fast Forward, web shortcuts, etc.) actually do something sensible, both on the desktop and in the context of the command line and any given application (e.g. watching a video). Handy links to descriptions of traditional behavior: http://en.wikipedia.org/wiki/Keyboard_(computing) http://en.wikipedia.org/wiki/Category:Computer_keys * Persistence - Do your desktop settings stay in place after you logout? After you reboot? What about numlock, capslock, etc? * Desktop Stress Test - Add lots of applets to a panel. Do sensible things happen when notification icons appear? Do all the applets work as advertised? Are any of them redundant? Open multiple documents in multiple applications (OpenOffice, Firefox, Emacs, Gimp, etc.). Does the default configuration do sensible things (it's easy to find the document you are looking for using the window list)? What about alternate window-navigation tools? What about if you add multiple workspaces to the mix? * Working Remotely - Do applications function properly if you are running them remotely via ssh or other tool? Some known problems are related to: https://bugzilla.redhat.com/show_bug.cgi?id=484171 (ssh doesn't forward dbus) Applications should function acceptably even given this limitation. * Crashes - All crashes should bring up a user-friendly assistant to report the problem. Bug reports should be successfully filed in a suitable bug tracker without any special knowledge on the part of the reporter. Crashes can be induced from the command line with "kill -SEGV". More info: http://fedoraproject.org/wiki/Features/ABRT (Some applications use different crash-handling software.) It would also be nice if applications recovered from crashes without significant data loss. (Can be tested by opening a document and making several changes without saving, waiting a little while to let any periodic auto-save kick in, then inducing a crash.) Obviously any real crashes that are encountered during testing should be reported, since they shouldn't happen in the first place. As testing days are finished, it might be interesting to output a not-very-detailed but growing list similar to http://fedoraproject.org/wiki/QA/ReleaseCriteria but for "things the user interface should do seamlessly". As in, "if it's not doing this for you, you should file a bug" and also as guide for future re-testing. Or maybe it's more efficient to leave tracking up to Bugzilla and just trust people to know something is wrong (whether it's a bug or a design flaw) when they see it. -B. -- fedora-test-list mailing list fedora-test-list@xxxxxxxxxx To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-test-list
