On Thu, 2011-01-20 at 05:54 -0500, Samuel Greenfeld wrote: +AD4 Unfortunately while I am familiar with a few test case management +AD4 systems, I have not been involved with the Fedora project long enough +AD4 to know its workflows. And a quick search online is not turning up +AD4 much about Nitrate, with its changelog+AFs-1+AF0 implying it was first made +AD4 open source software in July 2010. +AD4 +AD4 I might be able to provide some general comments though if I knew more +AD4 about the product. So for those of us unfamiliar with the history of +AD4 Nitrate, could you please answer the following: Really good list of questions Samuel, thanks for jumping in. I know Hurry can provide feedback on the questions. However, I'll add my thoughts as well. +AD4 1. What is the history of Nitrate and the Fedora Project? What +AD4 does the Fedora project expect to gain from using it? Hurry can touch on the goals for using nitrate. As for history, I can add my experiences ... This goes back to an eval we did using testopia in Fedora many releases agove. Unfortunately, the effort was canceled due to license incompatibility between Fedora and testopia. At that point, we invested in leveraging the wiki to best of our ability. https://fedoraproject.org/wiki/QA/Testopia+AF8-Evaluation https://bugzilla.redhat.com/show+AF8-bug.cgi?id+AD0-450013 After this, several folks got together and decided they would implement a new front-end on top of the testopia db schema. This would resolve the original license incompatibility and address usability issues that were raised with the testopia UI. The project started development internally, and was open-sourced in 2010. +AD4 1. Is a sample play/sandbox test instance with more-or-less full +AD4 access available online? Not yet, I believe that's in plan for sometime during Fedora 15. Hurry and I were discussing the requirements for such an instance earlier this week. If you're interested in helping here, just let us know :) +AD4 1. How does Nitrate compare to other open +ACY closed sourced TCMS +AD4 solutions? Why was it written as opposed to using an existing +AD4 solution, and what are its straights +ACY weaknesses? See history comments above. Also, maybe the nitrate developers +AFs-1+AF0 can offer more insight on how it compares to other open-source solutions? I +ACo-think+ACo that comparison work has been done in the past, I'm just not sure where to find the results. +AFs-1+AF0 https://fedorahosted.org/mailman/listinfo/nitrate-devel +AD4 1. Is test case +ACY test plan import and export (to XML, etc.) +AD4 support available? If so is this compatible with any other +AD4 TCMS's import/export system? I believe import/export is supported using the testopia.dtd format. http://git.fedorahosted.org/git/?p+AD0-nitrate.git+ADs-a+AD0-blob+ADs-f+AD0-trunk/nitrate/docs/ChangeLog - Fixed +ACM-564258 - +AFs-REF+AF0 Ability to export/print specified cases - Fixed +ACM-612803 - add an export feature for test case runs, can export +ICY +AD4 1. Are nested test plan and/or test case hierarchies supported? I don't recall. Perhaps Hurry or the nitrate developers know? I know this feature has been discussed a +ACo-lot+ACo with nitrate, and other solutions we've used in the past. I don't think support for nested test plans is something we've had a tremendous need for now, so I don't anticipate this being a MUSTHAVE feature in the near term. +AD4 1. Can multiple projects share test cases, and even reference +AD4 older versions of test cases if they are lagging behind the +AD4 current rawhide/Fedora release? Will Spins be able to make +AD4 their own (simultaneously running) test plans? This is the hope. It's not really useful if we can only use it for release validation. I don't think we've fully explored how best to model other spins/projects, but I don't foresee big problems there. That will be fun to explore on the sandbox/staging instance. With regards to referencing older versions of a test case, I believe that support is there, although I'm not certain it's right for our needs. Keeping test documentation (plans and cases) updated is a pretty sizable maintenance challenge. I've seen many instances where support for versioned test cases allows test plans to suffer over time as they were linked against old and inaccurate test cases. Much like how the wiki is used now, we have support for linking against older versions of tests (wiki history), but we rarely ever use that feature. I expect that trend would continue in the short-term. +AD4 1. How long will historical test case results be made available? I suspect the limiting factor here is database size. I'm not aware of any rules or process that would require removal/archival of old results. However, at some point that could certainly be an issue we'd need to plan for. +AD4 1. Is there any plan to tie this into Bodhi and other tools to +AD4 detect updated packages that may imply test cases need +AD4 re-running and/or updating? That is certainly possible, but there are currently no detailed plans to integrate bodhi/f-e-k with nitrate specifically. Nitrate offers a robust XMLRPC interface to allow other tools, such as bodhi, to query for applicable tests/results/runs etc... So I wouldn't think it would be a difficult task. That said, Adam Williamson is exploring this linkage using our current wiki-based test management system. While the two implementations will be completely different, the end result is the same ... improved integration between our update tools (bodhi/f-e-k) and our test infrastructure. +AD4 1. Is this going to be available as a Fedorahosted service like +AD4 Trac is? If so will all the instances be able to share test +AD4 cases? Hmm, I don't envision multiple hosted instances of nitrate (or any other test case management) in Fedora. I expect we would have a single test management system available for all to use, much like we have a build system, updates and bug reporting. +AD4 1. Is there any concern that changing test tracking systems may +AD4 encourage/discourage existing testers to participate? Yes+ACE That's part of why Hurry is investing a lot of time researching our current workflows and comparing features between our current wiki-based implementation. +AD4 Some of this information may be useful to post on the Trac main page +AD4 and/or in the Fedora wiki. +AD4 +AD4 Thanks in advance for your time. Thank you for your input, good comments/questions+ACE Thanks, James
Attachment:
signature.asc
Description: This is a digitally signed message part
-- test mailing list test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe: https://admin.fedoraproject.org/mailman/listinfo/test
