A few more inline comments on a subset of the questions, and two more thoughts: On Thu, Jan 20, 2011 at 11:54 AM, James Laska <jlaska@xxxxxxxxxx> wrote: > On Thu, 2011-01-20 at 05:54 -0500, Samuel Greenfeld wrote: > > 1. What is the history of Nitrate and the Fedora Project? What > > does the Fedora project expect to gain from using it? > > This goes back to an eval we did using testopia in Fedora many releases > agove. Unfortunately, the effort was canceled due to license > incompatibility between Fedora and testopia. At that point, we invested > in leveraging the wiki to best of our ability. > > https://fedoraproject.org/wiki/QA/Testopia+AF8-Evaluation > https://bugzilla.redhat.com/show_bug.cgi?id=450013 >From the bug I'm guessing this is because Testopia used Ext-JS, and Ext-JS kept changing licenses. Hopefully there will only be licensing restrictions for Nitrate on the software itself, and not the created work of what one does while actively using it. > > 1. How does Nitrate compare to other open & closed sourced TCMS > > solutions? Why was it written as opposed to using an existing > > solution, and what are its strengths & weaknesses? > > See history comments above. Also, maybe the nitrate developers [1] can > offer more insight on how it compares to other open-source solutions? I > *think* that comparison work has been done in the past, I'm just not > sure where to find the results. > > [1] https://fedorahosted.org/mailman/listinfo/nitrate-devel Are the developers actively monitoring/using this list? I looked at it before, and all I saw were three test posts from July in the archives. > > 1. Can multiple projects share test cases, and even reference > > older versions of test cases if they are lagging behind the > > current rawhide/Fedora release? Will Spins be able to make > > their own (simultaneously running) test plans? > > This is the hope. It's not really useful if we can only use it for > release validation. I don't think we've fully explored how best to > model other spins/projects, but I don't foresee big problems there. > That will be fun to explore on the sandbox/staging instance. > > With regards to referencing older versions of a test case, I believe > that support is there, although I'm not certain it's right for our > needs. Keeping test documentation (plans and cases) updated is a pretty > sizable maintenance challenge. I've seen many instances where support > for versioned test cases allows test plans to suffer over time as they > were linked against old and inaccurate test cases. > > Much like how the wiki is used now, we have support for linking against > older versions of tests (wiki history), but we rarely ever use that > feature. I expect that trend would continue in the short-term. The reason I bring this up is because OLPC's Spin releases tend to lag behind the official Fedora release. For instance, we just released our hopefully final Fedora 11-based release this past month, and are in the early stages of the Fedora 14-based one. At least some Fedora ARM development work may still be going on with Fedora 13 as well. While decently written test cases will survive somewhat over time, major changes in GUI look & feel or other areas can break their backwards compatibility. Ideally Nitrate will default to using the current version of a testcase when making a new plan, preferably following the updates of said testcase until a result is committed which forces the test case version to be needed. > > 1. How long will historical test case results be made available? > > I suspect the limiting factor here is database size. I'm not aware of > any rules or process that would require removal/archival of old results. > However, at some point that could certainly be an issue we'd need to > plan for. Again; this would be to help lagging Spins and similar. Right now it looks like Bodhi at least publicly hides information for Fedora versions which have reached end-of-life. (Either that, or I don't know how to find it.) I presume that Nitrate has already been determined to be scalable; otherwise it is a big risk to incorporate it into Fedora. Also: It might be useful to add an "unclear" testcase result, similar to how Mozilla's Litmus system does it (https://litmus.mozilla.org). -- test mailing list test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe: https://admin.fedoraproject.org/mailman/listinfo/test
