Re: Evaluating MozTrap as a TCMS for Fedora QA

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 08/14/2014 02:44 PM, Adam Williamson wrote:
On 2014-08-14 09:42, Amita Sharma wrote:

you missed one last question :: If this decision is made, have we
thought about the migration/porting of existing wiki test case/plans
to tcms?

Oops, indeed, sorry (missed it in the nesting).

That's one of the slightly trickier areas. Moztrap is bound to its 'one step, one expected response' model, where we have the %prep stage and we have separate stages for 'steps' and 'results' so they can have any kind of mapping, it doesn't have to be 1:1.

My initial thought was that we dump the steps from %prep into the test description wholesale (this is what upstream basically recommends - they say any preliminary details about test setup and so on should be in the descripton). For the step/result thing, what I figured was that we should just put in whatever we have and leave the 'corresponding' step or result empty for any imbalance (so if there are 5 steps and 3 expected results, you'd have 2 empty expected result sections, if there are 2 steps and 6 expected results, you'd have 4 empty step sections), then fix it up by hand - I don't think we can get any cleverer with the conversion. Even where the quantity of steps and results matches they won't necessarily correspond perfectly 1-to-1 in the way moztrap expects.

So basically I think this issue means we can't achieve a fully automated conversion, we'd need a degree of manual intervention / review for every test case. I don't think that's completely unmanageable, though, I believe our current test case corpus is in the hundreds not the thousands and a lot of them could do with review anyway (and we can always do a staged conversion - for instance only move validation testing to moztrap at first, then we'd only need to convert the validation test cases). It's something we could blow through as a hackfest (remote) type thing, if we chose.

So far as the tech goes, it's fairly trivial - moztrap supports a very lightly structured text format for imports, so you just have to write something which yoinks test case pages from mediawiki and does a fairly straightforward bit of text munging on them. Well, if we try to retain formatting in some way we can make the text munging more complex, it depends how much we care.

http://moztrap.readthedocs.org/en/latest/userguide/ui/import.html#bulk-test-case-entry-formats is the documentation for bulk entry; it gives the basic idea, there are some details that aren't covered (like if we use markdown syntax in the Gherkin-style import format, will it show up in the resulting test case? - that kinda thing).

We could of course contribute code to make moztrap not assume that all test cases have one result per step, but that's probably more work than just adjusting our test cases to that format.
I agree with the idea doing it with some automation + some manual intervention as a hackfest. So +1
I am not sure about contributing code for this :/ ( -1)

Thanks,
Ami
--
test mailing list
test@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test





[Index of Archives]     [Fedora Desktop]     [Fedora SELinux]     [Photo Sharing]     [Yosemite Forum]     [KDE Users]

  Powered by Linux